FAQ
上传用户:coffee44
上传日期:2018-10-23
资源大小:12304k
文件大小:49k
- Updated: Jan 14, 2009 (http://curl.haxx.se/docs/faq.html)
- _ _ ____ _
- ___| | | | _ | |
- / __| | | | |_) | |
- | (__| |_| | _ <| |___
- ___|___/|_| ______|
- FAQ
- 1. Philosophy
- 1.1 What is cURL?
- 1.2 What is libcurl?
- 1.3 What is curl not?
- 1.4 When will you make curl do XXXX ?
- 1.5 Who makes curl?
- 1.6 What do you get for making curl?
- 1.7 What about CURL from curl.com?
- 1.8 I have a problem who do I mail?
- 1.9 Where do I buy commercial support for curl?
- 1.10 How many are using curl?
- 1.11 Why don't you update ca-bundle.crt
- 2. Install Related Problems
- 2.1 configure doesn't find OpenSSL even when it is installed
- 2.1.1 native linker doesn't find OpenSSL
- 2.1.2 only the libssl lib is missing
- 2.2 Does curl work/build with other SSL libraries?
- 2.3 Where can I find a copy of LIBEAY32.DLL?
- 2.4 Does curl support Socks (RFC 1928) ?
- 3. Usage Problems
- 3.1 curl: (1) SSL is disabled, https: not supported
- 3.2 How do I tell curl to resume a transfer?
- 3.3 Why doesn't my posting using -F work?
- 3.4 How do I tell curl to run custom FTP commands?
- 3.5 How can I disable the Pragma: nocache header?
- 3.6 Does curl support ASP, XML, XHTML or HTML version Y?
- 3.7 Can I use curl to delete/rename a file through FTP?
- 3.8 How do I tell curl to follow HTTP redirects?
- 3.9 How do I use curl in my favorite programming language?
- 3.10 What about SOAP, WebDAV, XML-RPC or similar protocols over HTTP?
- 3.11 How do I POST with a different Content-Type?
- 3.12 Why do FTP specific features over HTTP proxy fail?
- 3.13 Why does my single/double quotes fail?
- 3.14 Does curl support javascript or pac (automated proxy config)?
- 3.15 Can I do recursive fetches with curl?
- 3.16 What certificates do I need when I use SSL?
- 3.17 How do I list the root dir of an FTP server?
- 3.18 Can I use curl to send a POST/PUT and not wait for a response?
- 4. Running Problems
- 4.1 Problems connecting to SSL servers.
- 4.2 Why do I get problems when I use & or % in the URL?
- 4.3 How can I use {, }, [ or ] to specify multiple URLs?
- 4.4 Why do I get downloaded data even though the web page doesn't exist?
- 4.5 Why do I get return code XXX from a HTTP server?
- 4.5.1 "400 Bad Request"
- 4.5.2 "401 Unauthorized"
- 4.5.3 "403 Forbidden"
- 4.5.4 "404 Not Found"
- 4.5.5 "405 Method Not Allowed"
- 4.5.6 "301 Moved Permanently"
- 4.6 Can you tell me what error code 142 means?
- 4.7 How do I keep user names and passwords secret in Curl command lines?
- 4.8 I found a bug!
- 4.9 Curl can't authenticate to the server that requires NTLM?
- 4.10 My HTTP request using HEAD, PUT or DELETE doesn't work!
- 4.11 Why does my HTTP range requests return the full document?
- 4.12 Why do I get "certificate verify failed" ?
- 4.13 Why is curl -R on Windows one hour off?
- 4.14 Redirects work in browser but not with curl!
- 4.15 FTPS doesn't work
- 4.16 My HTTP POST or PUT requests are slow!
- 5. libcurl Issues
- 5.1 Is libcurl thread-safe?
- 5.2 How can I receive all data into a large memory chunk?
- 5.3 How do I fetch multiple files with libcurl?
- 5.4 Does libcurl do Winsock initing on win32 systems?
- 5.5 Does CURLOPT_WRITEDATA and CURLOPT_READDATA work on win32 ?
- 5.6 What about Keep-Alive or persistent connections?
- 5.7 Link errors when building libcurl on Windows!
- 5.8 libcurl.so.3: open failed: No such file or directory
- 5.9 How does libcurl resolve host names?
- 5.10 How do I prevent libcurl from writing the response to stdout?
- 5.11 How do I make libcurl not receive the whole HTTP response?
- 5.12 Can I make libcurl fake or hide my real IP address?
- 5.13 How do I stop an ongoing transfer?
- 5.14 Using C++ non-static functions for callbacks?
- 6. License Issues
- 6.1 I have a GPL program, can I use the libcurl library?
- 6.2 I have a closed-source program, can I use the libcurl library?
- 6.3 I have a BSD licensed program, can I use the libcurl library?
- 6.4 I have a program that uses LGPL libraries, can I use libcurl?
- 6.5 Can I modify curl/libcurl for my program and keep the changes secret?
- 6.6 Can you please change the curl/libcurl license to XXXX?
- 6.7 What are my obligations when using libcurl in my commerical apps?
- 7. PHP/CURL Issues
- 7.1 What is PHP/CURL?
- 7.2 Who write PHP/CURL?
- 7.3 Can I perform multiple requests using the same handle?
- ==============================================================================
- 1. Philosophy
- 1.1 What is cURL?
- cURL is the name of the project. The name is a play on 'Client for URLs',
- originally with URL spelled in uppercase to make it obvious it deals with
- URLs. The fact it can also be pronounced 'see URL' also helped, it works as
- an abbreviation for "Client URL Request Library" or why not the recursive
- version: "Curl URL Request Library".
- The cURL project produces two products:
- libcurl
- A free and easy-to-use client-side URL transfer library, supporting FTP,
- FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, FILE, LDAP and LDAPS.
- libcurl supports HTTPS certificates, HTTP POST, HTTP PUT, FTP uploading,
- kerberos, HTTP form based upload, proxies, cookies, user+password
- authentication, file transfer resume, http proxy tunneling and more!
- libcurl is highly portable, it builds and works identically on numerous
- platforms, including Solaris, NetBSD, FreeBSD, OpenBSD, Darwin, HPUX,
- IRIX, AIX, Tru64, Linux, UnixWare, HURD, Windows, Amiga, OS/2, BeOs, Mac
- OS X, Ultrix, QNX, OpenVMS, RISC OS, Novell NetWare, DOS and more...
- libcurl is free, thread-safe, IPv6 compatible, feature rich, well
- supported and fast.
- curl
- A command line tool for getting or sending files using URL syntax.
- Since curl uses libcurl, it supports a range of common Internet protocols,
- currently including HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TFTP, LDAP, LDAPS,
- DICT, TELNET and FILE.
- We pronounce curl and cURL with an initial k sound: [kurl].
- There are numerous sub-projects and related projects that also use the word
- curl in the project names in various combinations, but you should take
- notice that this FAQ is directed at the command-line tool named curl (and
- libcurl the library), and may therefore not be valid for other curl-related
- projects. (There is however a small section for the PHP/CURL in this FAQ.)
- 1.2 What is libcurl?
- libcurl is a reliable and portable library which provides you with an easy
- interface to a range of common Internet protocols.
- You can use libcurl for free in your application, be it open source,
- commercial or closed-source.
- libcurl is most probably the most portable, most powerful and most often
- used C-based multi-platform file transfer library on this planet - be it
- open source or commercial.
- 1.3 What is curl not?
- Curl is not a wget clone. That is a common misconception. Never, during
- curl's development, have we intended curl to replace wget or compete on its
- market. Curl is targeted at single-shot file transfers.
- Curl is not a web site mirroring program. If you want to use curl to mirror
- something: fine, go ahead and write a script that wraps around curl to make
- it reality (like curlmirror.pl does).
- Curl is not an FTP site mirroring program. Sure, get and send FTP with curl
- but if you want systematic and sequential behavior you should write a
- script (or write a new program that interfaces libcurl) and do it.
- Curl is not a PHP tool, even though it works perfectly well when used from
- or with PHP (when using the PHP/CURL module).
- Curl is not a program for a single operating system. Curl exists, compiles,
- builds and runs under a wide range of operating systems, including all
- modern Unixes (and a bunch of older ones too), Windows, Amiga, BeOS, OS/2,
- OS X, QNX etc.
- 1.4 When will you make curl do XXXX ?
- We love suggestions of what to change in order to make curl and libcurl
- better. We do however believe in a few rules when it comes to the future of
- curl:
- * Curl -- the command line tool -- is to remain a non-graphical command line
- tool. If you want GUIs or fancy scripting capabilities, you should look
- for another tool that uses libcurl.
- * We do not add things to curl that other small and available tools already
- do very fine at the side. Curl's output is fine to pipe into another
- program or redirect to another file for the next program to interpret.
- * We focus on protocol related issues and improvements. If you wanna do more
- magic with the supported protocols than curl currently does, chances are
- big we will agree. If you wanna add more protocols, we may very well
- agree.
- * If you want someone else to make all the work while you wait for us to
- implement it for you, that is not a very friendly attitude. We spend a
- considerable time already on maintaining and developing curl. In order to
- get more out of us, you should consider trading in some of your time and
- efforts in return.
- * If you write the code, chances are bigger that it will get into curl
- faster.
- 1.5 Who makes curl?
- curl and libcurl are not made by any single individual. Daniel Stenberg is
- project leader and main developer, but other persons' submissions are
- important and crucial. Anyone can contribute and post their changes and
- improvements and have them inserted in the main sources (of course on the
- condition that developers agree on that the fixes are good).
- The full list of all contributors is found in the docs/THANKS file.
- curl is developed by a community, with Daniel at the wheel.
- 1.6 What do you get for making curl?
- Project cURL is entirely free and open. No person gets paid for developing
- (lib)curl. We do this voluntarily on our spare time.
- We get some help from companies. CAG Contactor hosts the curl web site, Haxx
- owns the curl web site's domain and sourceforge.net hosts project services
- we take advantage from, like the bug tracker. Also, some companies have
- sponsored certain parts of the development in the past and I hope some will
- continue to do so in the future.
- If you want to support our project, consider a donation or a banner-program
- or even better: by helping us coding, documenting, testing etc.
- 1.7 What about CURL from curl.com?
- During the summer 2001, curl.com was busy advertising their client-side
- programming language for the web, named CURL.
- We are in no way associated with curl.com or their CURL programming
- language.
- Our project name curl has been in effective use since 1998. We were not the
- first computer related project to use the name "curl" and do not claim any
- first-hand rights to the name.
- We recognize that we will be living in parallel with curl.com and wish them
- every success.
- 1.8 I have a problem who do I mail?
- Please do not mail any single individual unless you really need to. Keep
- curl-related questions on a suitable mailing list. All available mailing
- lists are listed in the MANUAL document and online at
- http://curl.haxx.se/mail/
- Keeping curl-related questions and discussions on mailing lists allows
- others to join in and help, to share their ideas, contribute their
- suggestions and spread their wisdom. Keeping discussions on public mailing
- lists also allows for others to learn from this (both current and future
- users thanks to the web based archives of the mailing lists), thus saving us
- from having to repeat ourselves even more. Thanks for respecting this.
- If you have found or simply suspect a security problem in curl or libcurl,
- mail curl-security at haxx.se (closed list of receivers, mails are not
- disclosed) and tell. Then we can produce a fix in a timely manner before the
- flaw is announced to the world, thus lessen the impact the problem will have
- on existing users.
- 1.9 Where do I buy commercial support for curl?
- curl is fully open source. It means you can hire any skilled engineer to fix
- your curl-related problems.
- We list available alternatives on the curl web site:
- http://curl.haxx.se/support.html
- 1.10 How many are using curl?
- It is impossible to tell.
- We don't know how many users that knowingly have installed and use curl.
- We don't know how many users that use curl without knowing that they are in
- fact using it.
- We don't know how many users that downloaded or installed curl and then
- never use it.
- Some facts to use as input to the math:
- curl packages are downloaded from the curl.haxx.se and mirrors over a
- million times per year. curl is installed by default with most Linux
- distributions. curl is installed by default with Mac OS X. curl and libcurl
- as used by numerous applications that include libcurl binaries in their
- distribution packages (like Adobe Acrobat Reader and Google Earth).
- More than 90 known named companies use curl in commercial environments and
- products. More than 100 known named open source projects depend on
- (lib)curl.
- In a poll on the curl web site mid-2005, more than 50% of the 300+ answers
- estimated a user base of one million users or more.
- In March 2005, the "Linux Counter project" estimated a total Linux user base
- of some 29 millions, while Netcraft detected some 4 million "active" Linux
- based web servers. A guess is that a fair amount of these Linux
- installations have curl installed.
- All this taken together, there is no doubt that there are millions of
- (lib)curl users.
- http://curl.haxx.se/docs/companies.html
- http://curl.haxx.se/docs/programs.html
- http://curl.haxx.se/libcurl/using/apps.html
- http://counter.li.org/estimates.php
- http://news.netcraft.com/archives/2005/03/14/fedora_makes_rapid_progress.html
- 1.11 Why don't you update ca-bundle.crt
- The ca-bundle.crt file that used to be bundled with curl was very outdated
- (it being last modified year 2000 should tell) and must be replaced with a
- much more modern and up-to-date version by anyone who wants to verify peers
- anyway. It is no longer provided, the last curl release that shipped it was
- curl 7.18.0.
- In the cURL project we've decided not to attempt to keep this file updated
- (or even present anymore) since deciding what to add to a ca cert bundle is
- an undertaking we've not been ready to accept, and the one we can get from
- Mozilla is perfectly fine so there's no need to duplicate that work.
- Today, with many services performed over HTTPS, every operating system
- should come with a default ca cert bundle that can be deemed somewhat
- trustworthy and that collection (if reasonably updated) should be deemed to
- be a lot better than a private curl version.
- If you want the most recent collection of ca certs that Mozilla Firefox
- uses, we recommend that you extract the collection yourself from Mozilla
- Firefox (by running 'make ca-bundle), or by using our online service setup
- for this purpose: http://curl.haxx.se/docs/caextract.html
- 2. Install Related Problems
- 2.1 configure doesn't find OpenSSL even when it is installed
- This may be because of several reasons.
- 2.1.1 native linker doesn't find openssl
- Affected platforms:
- Solaris (native cc compiler)
- HPUX (native cc compiler)
- SGI IRIX (native cc compiler)
- SCO UNIX (native cc compiler)
- When configuring curl, I specify --with-ssl. OpenSSL is installed in
- /usr/local/ssl Configure reports SSL in /usr/local/ssl, but fails to find
- CRYPTO_lock in -lcrypto
- Cause: The cc for this test places the -L/usr/local/ssl/lib AFTER
- -lcrypto, so ld can't find the library. This is due to a bug in the GNU
- autoconf tool.
- Workaround: Specifying "LDFLAGS=-L/usr/local/ssl/lib" in front of
- ./configure places the -L/usr/local/ssl/lib early enough in the command
- line to make things work
- 2.1.2 only the libssl lib is missing
- If all include files and the libcrypto lib is present, with only the
- libssl being missing according to configure, this is mostly likely because
- a few functions are left out from the libssl.
- If the function names missing include RSA or RSAREF you can be certain
- that this is because libssl requires the RSA and RSAREF libs to build.
- See the INSTALL file section that explains how to add those libs to
- configure. Make sure that you remove the config.cache file before you
- rerun configure with the new flags.
- 2.2 Does curl work/build with other SSL libraries?
- Curl has been written to use OpenSSL, GnuTLS, yassl or NSS, although there
- should not be many problems using a different library. If anyone does "port"
- curl to use a different SSL library, we are of course very interested in
- getting the patch!
- 2.3 Where can I find a copy of LIBEAY32.DLL?
- That is an OpenSSL binary built for Windows.
- Curl uses OpenSSL to do the SSL stuff. The LIBEAY32.DLL is what curl needs
- on a windows machine to do https://. Check out the curl web site to find
- accurate and up-to-date pointers to recent OpenSSL DLLs and other binary
- packages.
- 2.4 Does curl support Socks (RFC 1928) ?
- Yes, SOCKS 4 and 5 are supported.
- 3. Usage problems
- 3.1 curl: (1) SSL is disabled, https: not supported
- If you get this output when trying to get anything from a https:// server,
- it means that the configure script couldn't find all libs and include files
- it requires for SSL to work. If the configure script fails to find them,
- curl is simply built without SSL support.
- To get the https:// support into a curl that was previously built but that
- reports that https:// is not supported, you should dig through the document
- and logs and check out why the configure script doesn't find the SSL libs
- and/or include files.
- Also, check out the other paragraph in this FAQ labeled "configure doesn't
- find OpenSSL even when it is installed".
- 3.2 How do I tell curl to resume a transfer?
- Curl supports resumed transfers both ways on both FTP and HTTP.
- Try the -C option.
- 3.3 Why doesn't my posting using -F work?
- You can't simply use -F or -d at your choice. The web server that will
- receive your post assumes one of the formats. If the form you're trying to
- "fake" sets the type to 'multipart/form-data', then and only then you must
- use the -F type. In all the most common cases, you should use -d which then
- causes a posting with the type 'application/x-www-form-urlencoded'.
- This is described in some detail in the MANUAL and TheArtOfHttpScripting
- documents, and if you don't understand it the first time, read it again
- before you post questions about this to the mailing list. Also, try reading
- through the mailing list archives for old postings and questions regarding
- this.
- 3.4 How do I tell curl to run custom FTP commands?
- You can tell curl to perform optional commands both before and/or after a
- file transfer. Study the -Q/--quote option.
- Since curl is used for file transfers, you don't use curl to just perform
- FTP commands without transferring anything. Therefore you must always specify
- a URL to transfer to/from even when doing custom FTP commands.
- 3.5 How can I disable the Pragma: nocache header?
- You can change all internally generated headers by adding a replacement with
- the -H/--header option. By adding a header with empty contents you safely
- disable that one. Use -H "Pragma:" to disable that specific header.
- 3.6 Does curl support ASP, XML, XHTML or HTML version Y?
- To curl, all contents are alike. It doesn't matter how the page was
- generated. It may be ASP, PHP, Perl, shell-script, SSI or plain
- HTML-files. There's no difference to curl and it doesn't even know what kind
- of language that generated the page.
- See also item 3.14 regarding javascript.
- 3.7 Can I use curl to delete/rename a file through FTP?
- Yes. You specify custom FTP commands with -Q/--quote.
- One example would be to delete a file after you have downloaded it:
- curl -O ftp://download.com/coolfile -Q '-DELE coolfile'
- or rename a file after upload:
- curl -T infile ftp://upload.com/dir/ -Q "-RNFR infile" -Q "-RNTO newname"
- 3.8 How do I tell curl to follow HTTP redirects?
- Curl does not follow so-called redirects by default. The Location: header
- that informs the client about this is only interpreted if you're using the
- -L/--location option. As in:
- curl -L http://redirector.com
- Not all redirects are HTTP ones, see 4.14
- 3.9 How do I use curl in my favorite programming language?
- There exist many language interfaces/bindings for curl that integrates it
- better with various languages. If you are fluid in a script language, you
- may very well opt to use such an interface instead of using the command line
- tool.
- Find out more about which languages that support curl directly, and how to
- install and use them, in the libcurl section of the curl web site:
- http://curl.haxx.se/libcurl/
- In February 2007, there are interfaces available for the following
- languages: Ada95, Basic, C, C++, Ch, Cocoa, D, Dylan, Euphoria, Ferite,
- Gambas, glib/GTK+, Java, Lisp, Lua, Mono, .NET, Object-Pascal, O'Caml,
- Pascal, Perl, PHP, PostgreSQL, Python, R, Rexx, Ruby, Scheme, S-Lang,
- Smalltalk, SPL, Tcl, Visual Basic, Q, wxwidgets and XBLite. By the time you
- read this, additional ones may have appeared!
- 3.10 What about SOAP, WebDAV, XML-RPC or similar protocols over HTTP?
- Curl adheres to the HTTP spec, which basically means you can play with *any*
- protocol that is built on top of HTTP. Protocols such as SOAP, WEBDAV and
- XML-RPC are all such ones. You can use -X to set custom requests and -H to
- set custom headers (or replace internally generated ones).
- Using libcurl is of course just as fine and you'd just use the proper
- library options to do the same.
- 3.11 How do I POST with a different Content-Type?
- You can always replace the internally generated headers with -H/--header.
- To make a simple HTTP POST with text/xml as content-type, do something like:
- curl -d "datatopost" -H "Content-Type: text/xml" [URL]
- 3.12 Why do FTP specific features over HTTP proxy fail?
- Because when you use a HTTP proxy, the protocol spoken on the network will
- be HTTP, even if you specify a FTP URL. This effectively means that you
- normally can't use FTP specific features such as FTP upload and FTP quote
- etc.
- There is one exception to this rule, and that is if you can "tunnel through"
- the given HTTP proxy. Proxy tunneling is enabled with a special option (-p)
- and is generally not available as proxy admins usually disable tunneling to
- other ports than 443 (which is used for HTTPS access through proxies).
- 3.13 Why does my single/double quotes fail?
- To specify a command line option that includes spaces, you might need to
- put the entire option within quotes. Like in:
- curl -d " with spaces " url.com
- or perhaps
- curl -d ' with spaces ' url.com
- Exactly what kind of quotes and how to do this is entirely up to the shell
- or command line interpreter that you are using. For most unix shells, you
- can more or less pick either single (') or double (") quotes. For
- Windows/DOS prompts I believe you're forced to use double (") quotes.
- Please study the documentation for your particular environment. Examples in
- the curl docs will use a mix of both these ones as shown above. You must
- adjust them to work in your environment.
- Remember that curl works and runs on more operating systems than most single
- individuals have ever tried.
- 3.14 Does curl support javascript or pac (automated proxy config)?
- Many web pages do magic stuff using embedded javascript. Curl and libcurl
- have no built-in support for that, so it will be treated just like any other
- contents.
- .pac files are a netscape invention and are sometimes used by organizations
- to allow them to differentiate which proxies to use. The .pac contents is
- just a javascript program that gets invoked by the browser and that returns
- the name of the proxy to connect to. Since curl doesn't support javascript,
- it can't support .pac proxy configuration either.
- Some work-arounds usually suggested to overcome this javascript dependency:
- - Depending on the javascript complexity, write up a script that
- translates it to another language and execute that.
- - Read the javascript code and rewrite the same logic in another language.
- - Implement a javascript interpreter, people have successfully used the
- Mozilla javascript engine in the past.
- - Ask your admins to stop this, for a static proxy setup or similar.
- 3.15 Can I do recursive fetches with curl?
- No. curl itself has no code that performs recursive operations, such as
- those performed by wget and similar tools.
- There exist wrapper scripts with that functionality (for example the
- curlmirror perl script), and you can write programs based on libcurl to do
- it, but the command line tool curl itself cannot.
- 3.16 What certificates do I need when I use SSL?
- There are three different kinds of "certificates" to keep track of when we
- talk about using SSL-based protocols (HTTPS or FTPS) using curl or libcurl.
- - Client certificate. The server you communicate may require that you can
- provide this in order to prove that you actually are who you claim to be.
- If the server doesn't require this, you don't need a client certificate.
- - Server certificate. The server you communicate with has a server
- certificate. You can and should verify this certificate to make sure that
- you are truly talking to the real server and not a server impersonating
- it.
- - Certificate Authority certificate ("CA cert"). You often have several CA
- certs in a CA cert bundle that can be used to verify a server certificate
- that was signed by one of the authorities in the bundle. curl comes with a
- default CA cert bundle. You can override the default.
- The server certificate verification process is made by using a Certificate
- Authority certificate ("CA cert") that was used to sign the server
- certificate. Server certificate verification is enabled by default in curl
- and libcurl and is often the reason for problems as explained in FAQ entry
- 4.12 and the SSLCERTS document
- (http://curl.haxx.se/docs/sslcerts.html). Server certificates that are
- "self-signed" or otherwise signed by a CA that you do not have a CA cert
- for, cannot be verified. If the verification during a connect fails, you
- are refused access. You then need to explicitly disable the verification
- to connect to the server.
- 3.17 How do I list the root dir of an FTP server?
- There are two ways. The way defined in the RFC is to use an encoded slash
- in the first path part. List the "/tmp" dir like this:
- curl ftp://ftp.sunet.se/%2ftmp/
- or the not-quite-kosher-but-more-readable way, by simply starting the path
- section of the URL with a slash:
- curl ftp://ftp.sunet.se//tmp/
- 3.18 Can I use curl to send a POST/PUT and not wait for a response?
-
- No.
- But you could easily write your own program using libcurl to do such stunts.
- 4. Running Problems
- 4.1 Problems connecting to SSL servers.
- It took a very long time before we could sort out why curl had problems to
- connect to certain SSL servers when using SSLeay or OpenSSL v0.9+. The
- error sometimes showed up similar to:
- 16570:error:1407D071:SSL routines:SSL2_READ:bad mac decode:s2_pkt.c:233:
- It turned out to be because many older SSL servers don't deal with SSLv3
- requests properly. To correct this problem, tell curl to select SSLv2 from
- the command line (-2/--sslv2).
- There have also been examples where the remote server didn't like the SSLv2
- request and instead you had to force curl to use SSLv3 with -3/--sslv3.
- 4.2 Why do I get problems when I use & or % in the URL?
- In general unix shells, the & letter is treated special and when used, it
- runs the specified command in the background. To safely send the & as a part
- of a URL, you should quote the entire URL by using single (') or double (")
- quotes around it.
- An example that would invoke a remote CGI that uses &-letters could be:
- curl 'http://www.altavista.com/cgi-bin/query?text=yes&q=curl'
- In Windows, the standard DOS shell treats the %-letter specially and you
- need to use TWO %-letters for each single one you want to use in the URL.
- Also note that if you want the literal %-letter to be part of the data you
- pass in a POST using -d/--data you must encode it as '%25' (which then also
- needs the %-letter doubled on Windows machines).
- 4.3 How can I use {, }, [ or ] to specify multiple URLs?
- Because those letters have a special meaning to the shell, and to be used in
- a URL specified to curl you must quote them.
- An example that downloads two URLs (sequentially) would do:
- curl '{curl,www}.haxx.se'
- To be able to use those letters as actual parts of the URL (without using
- them for the curl URL "globbing" system), use the -g/--globoff option:
- curl -g 'www.site.com/weirdname[].html'
- 4.4 Why do I get downloaded data even though the web page doesn't exist?
- Curl asks remote servers for the page you specify. If the page doesn't exist
- at the server, the HTTP protocol defines how the server should respond and
- that means that headers and a "page" will be returned. That's simply how
- HTTP works.
- By using the --fail option you can tell curl explicitly to not get any data
- if the HTTP return code doesn't say success.
- 4.5 Why do I get return code XXX from a HTTP server?
- RFC2616 clearly explains the return codes. This is a short transcript. Go
- read the RFC for exact details:
- 4.5.1 "400 Bad Request"
- The request could not be understood by the server due to malformed
- syntax. The client SHOULD NOT repeat the request without modifications.
- 4.5.2 "401 Unauthorized"
- The request requires user authentication.
- 4.5.3 "403 Forbidden"
- The server understood the request, but is refusing to fulfill it.
- Authorization will not help and the request SHOULD NOT be repeated.
- 4.5.4 "404 Not Found"
- The server has not found anything matching the Request-URI. No indication
- is given of whether the condition is temporary or permanent.
- 4.5.5 "405 Method Not Allowed"
- The method specified in the Request-Line is not allowed for the resource
- identified by the Request-URI. The response MUST include an Allow header
- containing a list of valid methods for the requested resource.
- 4.5.6 "301 Moved Permanently"
- If you get this return code and an HTML output similar to this:
- <H1>Moved Permanently</H1> The document has moved <A
- HREF="http://same_url_now_with_a_trailing_slash/">here</A>.
- it might be because you request a directory URL but without the trailing
- slash. Try the same operation again _with_ the trailing URL, or use the
- -L/--location option to follow the redirection.
- 4.6 Can you tell me what error code 142 means?
- All curl error codes are described at the end of the man page, in the
- section called "EXIT CODES".
- Error codes that are larger than the highest documented error code means
- that curl has exited due to a crash. This is a serious error, and we
- appreciate a detailed bug report from you that describes how we could go
- ahead and repeat this!
- 4.7 How do I keep user names and passwords secret in Curl command lines?
- This problem has two sides:
- The first part is to avoid having clear-text passwords in the command line
- so that they don't appear in 'ps' outputs and similar. That is easily
- avoided by using the "-K" option to tell curl to read parameters from a file
- or stdin to which you can pass the secret info. curl itself will also
- attempt to "hide" the given password by blanking out the option - this
- doesn't work on all platforms.
- To keep the passwords in your account secret from the rest of the world is
- not a task that curl addresses. You could of course encrypt them somehow to
- at least hide them from being read by human eyes, but that is not what
- anyone would call security.
- Also note that regular HTTP (using Basic authentication) and FTP passwords
- are sent in clear across the network. All it takes for anyone to fetch them
- is to listen on the network. Eavesdropping is very easy. Use more secure
- authentication methods (like Digest, Negotiate or even NTLM) or consider the
- SSL-based alternatives HTTPS and FTPS.
- 4.8 I found a bug!
- It is not a bug if the behavior is documented. Read the docs first.
- Especially check out the KNOWN_BUGS file, it may be a documented bug!
- If it is a problem with a binary you've downloaded or a package for your
- particular platform, try contacting the person who built the package/archive
- you have.
- If there is a bug, read the BUGS document first. Then report it as described
- in there.
- 4.9 Curl can't authenticate to the server that requires NTLM?
- This is supported in curl 7.10.6 or later. No earlier curl version knows
- of this magic. Later versions require the OpenSSL or Microsoft Windows
- libraries to provide this functionality. Using GnuTLS or NSS libraries will
- not provide NTLM authentication functionality in curl.
- NTLM is a Microsoft proprietary protocol. Proprietary formats are evil. You
- should not use such ones.
- 4.10 My HTTP request using HEAD, PUT or DELETE doesn't work!
- Many web servers allow or demand that the administrator configures the
- server properly for these requests to work on the web server.
- Some servers seem to support HEAD only on certain kinds of URLs.
- To fully grasp this, try the documentation for the particular server
- software you're trying to interact with. This is not anything curl can do
- anything about.
- 4.11 Why does my HTTP range requests return the full document?
- Because the range may not be supported by the server, or the server may
- choose to ignore it and return the full document anyway.
- 4.12 Why do I get "certificate verify failed" ?
- You invoke curl 7.10 or later to communicate on a https:// URL and get an
- error back looking something similar to this:
- curl: (35) SSL: error:14090086:SSL routines:
- SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
- Then it means that curl couldn't verify that the server's certificate was
- good. Curl verifies the certificate using the CA cert bundle that comes with
- the curl installation.
- To disable the verification (which makes it act like curl did before 7.10),
- use -k. This does however enable man-in-the-middle attacks.
- If you get this failure but are having a CA cert bundle installed and used,
- the server's certificate is not signed by one of the CA's in the bundle. It
- might for example be self-signed. You then correct this problem by obtaining
- a valid CA cert for the server. Or again, decrease the security by disabling
- this check.
- Details are also in the SSLCERTS file in the release archives, found online
- here: http://curl.haxx.se/docs/sslcerts.html
- 4.13 Why is curl -R on Windows one hour off?
- During daylight savings time, when -R is used, curl will set a time that
- appears one hour off. This happens due to a flaw in how Windows stores and
- uses file modification times and it is not easily worked around. For details
- on this problem, read this: http://www.codeproject.com/datetime/dstbugs.asp
- 4.14 Redirects work in browser but not with curl!
- curl supports HTTP redirects fine (see item 3.8). Browsers generally support
- at least two other ways to perform directs that curl does not:
- - Meta tags. You can write a HTML tag that will cause the browser to
- redirect to another given URL after a certain time.
- - Javascript. You can write a javascript program embeded in a HTML page
- that redirects the browser to another given URL.
- There is no way to make curl follow these redirects. You must either
- manually figure out what the page is set to do, or you write a script that
- parses the results and fetches the new URL.
- 4.15 FTPS doesn't work
- curl supports FTPS (sometimes known as FTP-SSL) both implicit and explicit
- mode.
- When a URL is used that starts with FTPS://, curl assumes implicit SSL on
- the control connection and will therefore immediately connect and try to
- speak SSL. FTPS:// connections default to port 990.
- To use explicit FTPS, you use a FTP:// URL and the --ftp-ssl option (or one
- of its related flavours). This is the most common method, and the one
- mandated by RFC4217. This kind of connection then of course uses the
- standard FTP port 21 by default.
- 4.16 My HTTP POST or PUT requests are slow!
- libcurl makes all POST and PUT requests (except for POST requests with a
- very tiny request body) use the "Expect: 100-continue" header. This header
- allows the server to deny the operation early so that libcurl can bail out
- already before having to send any data. This is useful in authentication
- cases and others.
- However, many servers don't implement the Expect: stuff properly and if the
- server doesn't respond (positively) within 1 second libcurl will continue
- and send off the data anyway.
- You can disable libcurl's use of the Expect: header the same way you disable
- any header, using -H / CURLOPT_HTTPHEADER, or by forcing it to use HTTP 1.0.
- 5. libcurl Issues
- 5.1 Is libcurl thread-safe?
- Yes.
- We have written the libcurl code specifically adjusted for multi-threaded
- programs. libcurl will use thread-safe functions instead of non-safe ones if
- your system has such.
- If you use a OpenSSL-powered libcurl in a multi-threaded environment, you
- need to provide one or two locking functions:
- http://www.openssl.org/docs/crypto/threads.html#DESCRIPTION
- If you use a GnuTLS-powered libcurl in a multi-threaded environment, you
- need to provide locking function(s) for libgcrypt (which is used by GnuTLS
- for the crypto functions).
- http://www.gnu.org/software/gnutls/manual/html_node/Multi_002dthreaded-applications.html
- No special locking is needed with a NSS-powered libcurl. NSS is thread-safe.
- 5.2 How can I receive all data into a large memory chunk?
- [ See also the examples/getinmemory.c source ]
- You are in full control of the callback function that gets called every time
- there is data received from the remote server. You can make that callback do
- whatever you want. You do not have to write the received data to a file.
- One solution to this problem could be to have a pointer to a struct that you
- pass to the callback function. You set the pointer using the
- CURLOPT_WRITEDATA option. Then that pointer will be passed to the callback
- instead of a FILE * to a file:
- /* imaginary struct */
- struct MemoryStruct {
- char *memory;
- size_t size;
- };
- /* imaginary callback function */
- size_t
- WriteMemoryCallback(void *ptr, size_t size, size_t nmemb, void *data)
- {
- size_t realsize = size * nmemb;
- struct MemoryStruct *mem = (struct MemoryStruct *)data;
- mem->memory = (char *)realloc(mem->memory, mem->size + realsize + 1);
- if (mem->memory) {
- memcpy(&(mem->memory[mem->size]), ptr, realsize);
- mem->size += realsize;
- mem->memory[mem->size] = 0;
- }
- return realsize;
- }
- 5.3 How do I fetch multiple files with libcurl?
- libcurl has excellent support for transferring multiple files. You should
- just repeatedly set new URLs with curl_easy_setopt() and then transfer it
- with curl_easy_perform(). The handle you get from curl_easy_init() is not
- only reusable, but you're even encouraged to reuse it if you can, as that
- will enable libcurl to use persistent connections.
- 5.4 Does libcurl do Winsock initialization on win32 systems?
- Yes, if told to in the curl_global_init() call.
- 5.5 Does CURLOPT_WRITEDATA and CURLOPT_READDATA work on win32 ?
- Yes, but you cannot open a FILE * and pass the pointer to a DLL and have
- that DLL use the FILE * (as the DLL and the client application cannot access
- each others' variable memory areas). If you set CURLOPT_WRITEDATA you must
- also use CURLOPT_WRITEFUNCTION as well to set a function that writes the
- file, even if that simply writes the data to the specified FILE *.
- Similarly, if you use CURLOPT_READDATA you must also specify
- CURLOPT_READFUNCTION.
- 5.6 What about Keep-Alive or persistent connections?
- curl and libcurl have excellent support for persistent connections when
- transferring several files from the same server. Curl will attempt to reuse
- connections for all URLs specified on the same command line/config file, and
- libcurl will reuse connections for all transfers that are made using the
- same libcurl handle.
- 5.7 Link errors when building libcurl on Windows!
- You need to make sure that your project, and all the libraries (both static
- and dynamic) that it links against, are compiled/linked against the same run
- time library.
- This is determined by the /MD, /ML, /MT (and their corresponding /M?d)
- options to the command line compiler. /MD (linking against MSVCRT dll) seems
- to be the most commonly used option.
- When building an application that uses the static libcurl library, you must
- add -DCURL_STATICLIB to your CFLAGS. Otherwise the linker will look for
- dynamic import symbols. If you get linker error like "unknown symbol
- __imp__curl_easy_init ..." you have linked against the wrong (static)
- library. If you want to use the libcurl.dll and import lib, you don't need
- any extra CFLAGS, but use one of the import libraries below. These are the
- libraries produced by the various lib/Makefile.* files:
- Target: static lib. import lib for libcurl*.dll.
- -----------------------------------------------------------
- MingW: libcurl.a libcurldll.a
- MSVC (release): libcurl.lib libcurl_imp.lib
- MSVC (debug): libcurld.lib libcurld_imp.lib
- Borland: libcurl.lib libcurl_imp.lib
- 5.8 libcurl.so.3: open failed: No such file or directory
- This is an error message you might get when you try to run a program linked
- with a shared version of libcurl and your run-time linker (ld.so) couldn't
- find the shared library named libcurl.so.3.
- You need to make sure that ld.so finds libcurl.so.3. You can do that
- multiple ways, and it differs somewhat between different operating systems,
- but they are usually:
- * Add an option to the linker command line that specify the hard-coded path
- the run-time linker should check for the lib (usually -R)
- * Set an environment variable (LD_LIBRARY_PATH for example) where ld.so
- should check for libs
- * Adjust the system's config to check for libs in the directory where you've
- put the dir (like Linux's /etc/ld.so.conf)
- 'man ld.so' and 'man ld' will tell you more details
- 5.9 How does libcurl resolve host names?
- libcurl supports a large a number of different name resolve functions. One
- of them is picked at build-time and will be used unconditionally. Thus, if
- you want to change name resolver function you must rebuild libcurl and tell
- it to use a different function.
- - The non-ipv6 resolver that can use one out of four host name resolve calls
- (depending on what your system supports):
- A - gethostbyname()
- B - gethostbyname_r() with 3 arguments
- C - gethostbyname_r() with 5 arguments
- D - gethostbyname_r() with 6 arguments
- - The ipv6-resolver that uses getaddrinfo()
- - The c-ares based name resolver that uses the c-ares library for resolves.
- Using this offers asynchronous name resolves but it currently has no IPv6
- support.
- - The Windows threaded resolver. It use:
- A - gethostbyname() on plain ipv4 windows hosts
- B - getaddrinfo() on ipv6-enabled windows hosts
- Also note that libcurl never resolves or reverse-lookups addresses given as
- pure numbers, such as 127.0.0.1 or ::1.
- 5.10 How do I prevent libcurl from writing the response to stdout?
- libcurl provides a default built-in write function that writes received data
- to stdout. Set the CURLOPT_WRITEFUNCTION to receive the data, or possibly
- set CURLOPT_WRITEDATA to a different FILE * handle.
- 5.11 How do I make libcurl not receive the whole HTTP response?
- You make the write callback (or progress callback) return an error and
- libcurl will then abort the transfer.
- 5.12 Can I make libcurl fake or hide my real IP address?
- No. libcurl operates on a higher level than so. Besides, faking IP address
- would imply sending IP packages with a made-up source address, and then you
- normally get a problem with intercepting the packages sent back as they
- would then not be routed to you!
- If you use a proxy to access remote sites, the sites will not see your local
- IP address but instead the address of the proxy.
- Also note that on many networks NATs or other IP-munging techniques are used
- that makes you see and use a different IP address locally than what the
- remote server will see you coming from.
- 5.13 How do I stop an ongoing transfer?
- There are several ways, but none of them are instant. There is no function
- you can call from another thread or similar that will stop it immediately.
- Instead you need to make sure that one of the callbacks you use return an
- appropriate value that will stop the transfer.
- Suitable callbacks that you can do this with include the progress callback,
- the read callback and the write callback.
- If you're using the multi interface, you also stop a transfer by removing
- the particular easy handle from the multi stack.
- 5.14 Using C++ non-static functions for callbacks?
- libcurl is a C library, it doesn't know anything about C++ member functions.
- You can overcome this "limitation" with a relative ease using a static
- member function that is passed a pointer to the class:
- // f is the pointer to your object.
- static YourClass::staticFunction(void *buffer, size_t sz, size_t n, void *f)
- {
- // Call non-static member function.
- static_cast<YourClass*>(f)->nonStaticFunction();
- }
- // This is how you pass pointer to the static function:
- curl_easy_setopt(hcurl, CURLOPT_WRITEFUNCTION, YourClass:staticFunction);
- curl_easy_setopt(hcurl, CURLOPT_WRITEDATA, this);
- 6. License Issues
- Curl and libcurl are released under a MIT/X derivate license. The license is
- very liberal and should not impose a problem for your project. This section
- is just a brief summary for the cases we get the most questions. (Parts of
- this section was much enhanced by Bjorn Reese.)
- We are not lawyers and this is not legal advice. You should probably consult
- one if you want true and accurate legal insights without our prejudice.
- 6.1 I have a GPL program, can I use the libcurl library?
- Yes!
- Since libcurl may be distributed under the MIT/X derivate license, it can be
- used together with GPL in any software.
- 6.2 I have a closed-source program, can I use the libcurl library?
- Yes!
- libcurl does not put any restrictions on the program that uses the library.
- 6.3 I have a BSD licensed program, can I use the libcurl library?
- Yes!
- libcurl does not put any restrictions on the program that uses the library.
- 6.4 I have a program that uses LGPL libraries, can I use libcurl?
- Yes!
- The LGPL license doesn't clash with other licenses.
- 6.5 Can I modify curl/libcurl for my program and keep the changes secret?
- Yes!
- The MIT/X derivate license practically allows you to do almost anything with
- the sources, on the condition that the copyright texts in the sources are
- left intact.
- 6.6 Can you please change the curl/libcurl license to XXXX?
- No.
- We have carefully picked this license after years of development and
- discussions and a large amount of people have contributed with source code
- knowing that this is the license we use. This license puts the restrictions
- we want on curl/libcurl and it does not spread to other programs or
- libraries that use it. It should be possible for everyone to use libcurl or
- curl in their projects, no matter what license they already have in use.
- 6.7 What are my obligations when using libcurl in my commerical apps?
- Next to none. All you need to adhere to is the MIT-style license (stated in
- the COPYING file) which basically says you have to include the copyright
- notice in "all copies" and that you may not use the copyright holder's name
- when promoting your software.
- You do not have to release any of your source code.
- You do not have to reveal or make public any changes to the libcurl source
- code.
- You do not have to reveal or make public that you are using libcurl within
- your app.
- As can be seen here: http://curl.haxx.se/docs/companies.html and
- elsewhere, more and more companies are dicovering the power
- of libcurl and take advantage of it even in commercial environments.
- 7. PHP/CURL Issues
- 7.1 What is PHP/CURL?
- The module for PHP that makes it possible for PHP programs to access curl-
- functions from within PHP.
- In the cURL project we call this module PHP/CURL to differentiate it from
- curl the command line tool and libcurl the library. The PHP team however
- does not refer to it like this (for unknown reasons). They call it plain
- CURL (often using all caps) or sometimes ext/curl, but both cause much
- confusion to users which in turn gives us a higher question load.
- 7.2 Who write PHP/CURL?
- PHP/CURL is a module that comes with the regular PHP package. It depends and
- uses libcurl, so you need to have libcurl installed properly first before
- PHP/CURL can be used. PHP/CURL was initially written by Sterling Hughes.
- 7.3 Can I perform multiple requests using the same handle?
- Yes - at least in PHP version 4.3.8 and later (this has been known to not
- work in earlier versions, but the exact version when it started to work is
- unknown to me).
- After a transfer, you just set new options in the handle and make another
- transfer. This will make libcurl to re-use the same connection if it can.