pv

Print the contents of the file and display a progress bar

$ pv [path/to/file]

$ [command1] | pv [[-s|--size]] [expected_amount_of_data_for_eta] | [command2]

$ pv [[-cN|--cursor --name]] in [path/to/file.txt] | grep [pattern] | pv [[-cN|--cursor --name]] out > [path/to/filtered_file.txt]

$ pv [[-d|--watchfd]] [PID]

$ pv [[-EE|--skip-errors --skip-errors]] [path/to/faulty_media] > [path/to/image.img]

$ pv [[-L|--rate-limit]] [1K] [[-S|--stop-at-size]] [maximum_file_size_to_be_read]

$ pv [path/to/file] | [sha256sum]

pv [options] [file...]
command1 | pv [options] | command2
pv [options] < inputfile > outputfile

-s SIZE
    Sets the total data size to SIZE bytes. This is crucial for calculating percentage complete and ETA, especially when pv cannot determine the size from its input (e.g., from a pipe).

-L RATE
    Limits the transfer rate to RATE bytes per second. Useful for throttling bandwidth or disk I/O.

-p, -t, -e, -r, -b
    These flags control the display components: progress bar (-p), time elapsed (-t), ETA (-e), transfer rate (-r), and bytes transferred (-b). Most are displayed by default when pv is used interactively.

-q
    Quiet mode. Suppresses all progress output. Useful when pv is used solely for rate limiting (with -L) without visual feedback.

-d PID
    Instead of reading from standard input, monitors the file descriptors of a specific process ID (PID). Can be used to attach to running dd or cp commands.

-W
    Wait for input. Normally, pv exits if its input file reaches EOF. This option makes pv wait for more input instead, which is useful for monitoring growing files or ongoing streams.

-F FORMAT
    Allows specifying a custom output format string, providing fine-grained control over the displayed progress information and its layout.

pv (Pipe Viewer) is a lightweight, terminal-based utility designed to monitor the progress of data flowing through a pipeline. It acts as a transparent filter, reading data from its standard input and writing it to its standard output, while simultaneously displaying a visual indication of the data's transfer.

The progress information, which includes a progress bar, the time elapsed, the current data transfer rate, the total amount of data transferred, and an estimated time of arrival (ETA), is displayed to standard error, ensuring it doesn't interfere with the data stream itself. This makes pv an invaluable tool for operations involving large files or lengthy processes where visual feedback is crucial, such as copying files, creating archives, or processing data streams.

It is commonly integrated into shell scripts and command-line operations alongside tools like dd, tar, cat, cp, and gzip. By inserting pv into a pipeline (e.g., command1 | pv | command2), users gain real-time insight into the progress of data transfer, helping to gauge remaining time and identify potential bottlenecks. It can also be used to limit the data transfer rate, useful for throttling network or disk I/O.

For pv to provide accurate percentage and ETA estimates, especially when used in a pipe, it often needs to know the total size of the data to be processed. This can be specified using the -s SIZE option. If the total size is unknown, pv will only display elapsed time, transfer rate, and bytes transferred, but not percentage or ETA.

The progress output is directed to standard error (stderr), ensuring it doesn't interfere with the actual data stream passing from standard input to standard output. This means if you redirect stderr (e.g., to /dev/null), you might lose the progress output.

pv operates as a simple pass-through filter. It reads data from its standard input (stdin), passes that data unchanged to its standard output (stdout), and simultaneously sends progress updates to its standard error (stderr). This design allows it to be seamlessly inserted into any pipeline without affecting the data content or its integrity. It does not buffer or store the data itself, acting purely as a monitor.

To copy a large file with a progress bar and rate limit, you might use:

cat largefile.img | pv -L 10m -s $(stat -c%s largefile.img) > newfile.img

Here, -L 10m limits the transfer to 10 megabytes per second, and -s $(stat -c%s largefile.img) provides the total size of the input file to pv for accurate percentage and ETA calculation.

For copying raw disk images with progress:

dd if=/dev/sda | pv -s 500G | dd of=/dev/sdb

Or to monitor a network download with wget:

wget -O - http://example.com/largefile.iso | pv > largefile.iso

pv (Pipe Viewer) was created by Andrew Wood to address a common need in Unix-like environments: providing visual feedback for long-running data transfer operations, which traditionally offer no progress indication. Since its initial release, it has become a widely adopted and popular tool for enhancing the user experience of command-line operations involving data pipelines, filling a niche that other core utilities do not natively cover.

dd(1), cat(1), tar(1), progress(1)