pbzip2
parallel implementation of bzip2 compression
TLDR
Compress file
SYNOPSIS
pbzip2 [-d] [-k] [-p n] [-1..9] [options] [files]
DESCRIPTION
pbzip2 is a parallel implementation of bzip2 compression. It uses multiple CPU cores to compress and decompress files faster than single-threaded bzip2.
The tool produces files compatible with standard bzip2. Output can be decompressed by bzip2, bunzip2, or pbzip2 itself.
Parallelism scales with available processors. By default, all cores are used. The -p flag limits processor usage for background compression.
Compression levels work like bzip2: higher numbers produce smaller files but take longer. The parallel implementation helps offset slower compression speeds.
Memory usage scales with thread count. Each thread needs memory for its compression buffer. Limits can prevent memory exhaustion on large jobs.
Pipe mode supports streaming compression. Combined with tar, it enables parallel compressed archives.
PARAMETERS
-d
Decompress.-z
Compress (default).-k
Keep original files.-p N
Number of processors to use.-1 to -9
Compression level (9 = best).-c
Output to stdout.-f
Force overwrite.-t
Test integrity.-q
Quiet mode.-v
Verbose mode.-m N
Memory limit per thread (MB).-r
Read entire file into RAM.
CAVEATS
Memory usage higher than bzip2. Best speedup on multi-core systems. Decompression speedup depends on how file was created.
HISTORY
pbzip2 was created by Jeff Gilchrist around 2005 to leverage multi-core processors for bzip2 compression. As multi-core CPUs became common, parallel compression tools like pbzip2 became essential for handling large data.
