lwp-request

Make a simple GET request

$ lwp-request -m GET [http://example.com/some/path]

$ lwp-request -m POST [http://example.com/some/path] < [path/to/file]

$ lwp-request -H 'User-Agent: [user_agent] -m [METHOD] [http://example.com/some/path]

$ lwp-request -C [username]:[password] -m [METHOD] [http://example.com/some/path]

$ lwp-request -U -m [METHOD] [http://example.com/some/path]

$ lwp-request -E -m [METHOD] [http://example.com/some/path]

lwp-request [options] URL [URL...]
lwp-request -m method [options] URL [URL...]

-m method
    Specify the HTTP request method to use (e.g., GET, POST, HEAD, PUT, DELETE).

-d data
    Provide data to send with POST or PUT requests. Can be used multiple times for form data.

-H header:value
    Add a custom HTTP header to the request. Can be used multiple times.

-u user:pass
    Specify username and password for basic authentication.

-U user-agent
    Set the User-Agent string for the request, overriding the default.

-i
    Display only the response headers, without the content body.

-s
    Display the HTTP status line (e.g., 'HTTP/1.1 200 OK') before the content or headers.

-p proxy
    Specify a proxy server to use for the request (e.g., http://proxy.example.com:8080).

-L
    Follow HTTP redirects automatically (this is often the default behavior).

-z
    Do not follow HTTP redirects; return the response from the initial URL.

-t seconds
    Set a timeout for the request in seconds.

-k
    Do not verify SSL certificates; this is insecure and should only be used for testing or trusted environments.

lwp-request is a versatile command-line utility provided as part of the libwww-perl (LWP) library. It acts as a client for making various types of requests over the network, primarily HTTP and HTTPS, but also supports FTP and local files. It provides a simple interface to the robust functionalities of the LWP Perl module, allowing users to fetch web pages, submit form data (POST), inspect headers (HEAD), upload files (PUT), or delete resources (DELETE). This tool is invaluable for scripting web interactions, debugging web services, testing APIs, and performing quick fetches directly from the terminal without writing a full Perl script. It supports features like setting custom headers, handling redirects, authentication, and proxy configuration.

lwp-request relies on the libwww-perl library, which may not be pre-installed on all systems. While powerful, for very complex HTTP/HTTPS interactions or extremely high performance, dedicated tools like curl might offer more granular control or better optimized handling of specific protocols. Using the -k option (disabling SSL certificate verification) should be done with caution and only in trusted environments, as it bypasses critical security checks.

If lwp-request is not available on your system, it can usually be installed via your distribution's package manager. For Debian/Ubuntu, it's typically part of the libwww-perl package (e.g., sudo apt-get install libwww-perl). On Red Hat/CentOS, it might be perl-libwww-perl (e.g., sudo yum install perl-libwww-perl or sudo dnf install perl-libwww-perl).

Fetch a web page:
lwp-request http://example.com

Send a POST request with data:
lwp-request -m POST -d 'name=test&value=123' http://example.com/api

Get only response headers:
lwp-request -i http://example.com

Set a custom User-Agent:
lwp-request -U 'MyCustomBrowser/1.0' http://example.com

lwp-request emerged as a utility bundled with the libwww-perl (LWP) project, which started in the mid-1990s. LWP quickly became the de-facto standard for programmatic web access in Perl, providing a robust and flexible framework for HTTP, FTP, and other protocols. The lwp-request script was designed to offer a simple command-line wrapper around the core LWP functionalities, allowing developers and system administrators to quickly interact with web resources without writing full Perl programs. Its development has mirrored the evolution of web standards, continually updated to support new HTTP features and security protocols.

curl(1), wget(1), perl(1)