LinuxCommandLibrary

pee

Pipe input to multiple commands simultaneously

TLDR

Run each command, providing each one with a distinct copy of stdin

$ pee [command1 command2 ...]
copy

Write a copy of stdin to stdout (like tee)
$ pee cat [command1 command2 ...]
copy

Immediately terminate upon SIGPIPEs and write errors
$ pee --no-ignore-sigpipe --no-ignore-write-errors [command1 command2 ...]
copy

SYNOPSIS

pee COMMAND [ARGUMENTS] ... [COMMAND [ARGUMENTS] ...]

DESCRIPTION

pee is a utility from the moreutils package, designed to read its standard input and simultaneously pipe it to the standard input of multiple commands specified on its command line. Unlike tee(1) which duplicates input to files or standard output, pee duplicates input to other running processes. After feeding the input, pee then collects and interleaves the standard output from all these child commands, writing the combined output to its own standard output. This makes pee exceptionally useful for scenarios where the same dataset needs to be processed by several different, independent pipelines in parallel, and their collective results are desired. pee waits for all child processes to complete their execution before it exits. If any of the child commands terminate with a non-zero exit status, pee itself will also exit with a non-zero status, indicating a failure in one or more of the parallel operations.

CAVEATS

Output Interleaving: The standard output from the child commands is interleaved. If the order of output from different commands is critical, pee might not be suitable, or additional processing (e.g., sorting) might be required.
Resource Consumption: Running multiple commands concurrently can lead to higher CPU, memory, and I/O utilization compared to sequential processing.
Error Indication: pee exits with a non-zero status if any child command fails, but it does not specify which one failed or provide individual error codes from children.

EXAMPLE USAGE

echo "hello world" | pee "grep -o o" "tr 'oe' 'EO'"
(Possible output: E
O
o
o)

This example pipes "hello world" to two commands in parallel. The first grep extracts 'o's, and the second tr replaces 'o' and 'e' with 'E' and 'O' respectively. The output is interleaved.

ls | pee "wc -l" "grep -c .conf"
This command pipes the output of ls to wc -l (counting lines/files) and grep -c .conf (counting config files) simultaneously.

EXIT STATUS

pee's exit status is 0 if all child commands execute successfully. It returns a non-zero status if one or more child commands exit with a non-zero status, indicating a failure.

HISTORY

pee is part of the moreutils collection, a set of small, useful, and commonly needed Unix utilities that are often missing from standard distributions. Its development stemmed from the need for a "parallel tee" – a tool that could efficiently duplicate input streams to multiple separate processing pipelines simultaneously. It fills a niche for simple parallelization scenarios where direct piping to multiple commands is more straightforward than complex job schedulers or explicit parallel execution frameworks, emphasizing ease of use for common parallel stream processing tasks.

SEE ALSO

tee(1), xargs(1), parallel(1), mkfifo(1)

Copied to clipboard