LinuxCommandLibrary

curl

Transfer data from or to a server

TLDR

Make an HTTP GET request and dump the contents in stdout

$ curl [https://example.com]
copy

Make an HTTP GET request, follow any 3xx redirects, and dump the reply headers and contents to stdout
$ curl [[-L|--location]] [[-D|--dump-header]] - [https://example.com]
copy

Download a file, saving the output under the filename indicated by the URL
$ curl [[-O|--remote-name]] [https://example.com/filename.zip]
copy

Send form-encoded data (POST request of type application/x-www-form-urlencoded). Use --data @file_name or --data @'-' to read from stdin
$ curl [[-X|--request]] POST [[-d|--data]] ['name=bob'] [http://example.com/form]
copy

Send a request with an extra header, using a custom HTTP method and over a proxy (such as BurpSuite), ignoring insecure self-signed certificates
$ curl [[-k|--insecure]] [[-x|--proxy]] [http://127.0.0.1:8080] [[-H|--header]] ['Authorization: Bearer token'] [[-X|--request]] [GET|PUT|POST|DELETE|PATCH|...] [https://example.com]
copy

Send data in JSON format, specifying the appropriate Content-Type header
$ curl [[-d|--data]] ['{"name":"bob"}'] [[-H|--header]] ['Content-Type: application/json'] [http://example.com/users/1234]
copy

Pass client certificate and key for a resource, skipping certificate validation
$ curl [[-E|--cert]] [client.pem] --key [key.pem] [[-k|--insecure]] [https://example.com]
copy

Resolve a hostname to a custom IP address, with verbose output (similar to editing the /etc/hosts file for custom DNS resolution)
$ curl [[-v|--verbose]] --resolve [example.com]:[80]:[127.0.0.1] [http://example.com]
copy

SYNOPSIS

curl [options] [URL...]

Examples:
curl https://example.com
curl -O https://example.com/file.zip
curl -X POST -d "key=value" https://api.example.com/data

PARAMETERS

-o <file>, --output <file>
    Writes output to <file> instead of standard output.

-O, --remote-name
    Writes output to a file named as the remote file (e.g., if downloading from http://example.com/doc.pdf, saves as doc.pdf).

-L, --location
    Follows HTTP 3xx redirects.

-I, --head
    Fetches only the HTTP header information of a document.

-X <command>, --request <command>
    Specifies a custom request method to use when communicating with the HTTP server (e.g., GET, POST, PUT, DELETE).

-d <data>, --data <data>
    Sends the specified data in a POST request to the HTTP server. Often used for form submissions or sending JSON payloads.

-H <header>, --header <header>
    Passes a custom header to the server (e.g., 'Content-Type: application/json', 'Authorization: Bearer token').

-u <user:password>, --user <user:password>
    Specifies user and password for server authentication.

-k, --insecure
    Allows curl to proceed and operate even if the server's SSL certificate is invalid or cannot be verified. Use with caution.

-s, --silent
    Operates in silent or quiet mode, suppressing the progress meter and error messages.

-v, --verbose
    Makes the operation more talkative, displaying a lot of debugging information.

DESCRIPTION

curl is a robust and widely-used command-line tool for transferring data to or from a server. It supports an extensive array of protocols, including HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TFTP, DICT, TELNET, LDAP, LDAPS, FILE, IMAP, POP3, SMTP, and many more.

Developed to work without user interaction, curl is ideal for automated tasks, scripts, and applications. Its capabilities include support for proxies, user authentication, HTTP POST, SSL certificates, cookies, file transfer resume, and Metalink. It's an indispensable tool for developers for testing REST APIs, system administrators for downloading files and managing remote resources, and anyone needing to interact with web services or transfer data over a network.

CAVEATS

When using -k or --insecure, curl will bypass SSL certificate verification, which can expose your connection to security risks like Man-in-the-Middle attacks. It should be used only in development or controlled environments.

Be mindful of shell escaping when providing complex URLs or data payloads, especially with special characters or spaces, as the shell might interpret them before curl receives them.

For very large file transfers, consider using -o to stream directly to a file to avoid potential memory issues with very large in-memory buffers.

LIBCURL AND ITS IMPACT

Beyond being a command-line tool, curl is built upon the highly portable and powerful libcurl library. This library is widely used in countless software applications, devices, and operating systems to handle network communications, making curl's functionality accessible to developers for integration into their own projects.

COMMON USE CASES

curl is frequently used for:

  • Downloading files from remote servers.
  • Interacting with RESTful APIs (e.g., sending JSON data, fetching XML).
  • Debugging network requests and responses.
  • Testing web server configurations.
  • Automating data transfer in scripts and cron jobs.

HISTORY

curl was first released in 1997 by Daniel Stenberg under the name httpget. Its original purpose was to fetch currency exchange rates for an IRC bot. As its capabilities expanded beyond just HTTP GET requests, it was renamed to curl in 1998, a portmanteau of 'Client for URLs'.

The underlying library, libcurl, was subsequently developed, allowing applications to integrate curl's powerful data transfer capabilities. Over the years, curl has grown to support an impressive number of protocols and features, becoming an almost universal tool for network-related tasks across various operating systems.

SEE ALSO

wget(1), httpie(1), netcat(1), scp(1), sftp(1)

Copied to clipboard