LinuxCommandLibrary

pbzip2

parallel implementation of bzip2 compression

TLDR

Compress file

$ pbzip2 [file]
copy
Decompress file
$ pbzip2 -d [file.bz2]
copy
Keep original file
$ pbzip2 -k [file]
copy
Set compression level
$ pbzip2 -[9] [file]
copy
Use specific number of processors
$ pbzip2 -p[4] [file]
copy
Compress from stdin
$ cat [file] | pbzip2 > [file.bz2]
copy
Decompress to stdout
$ pbzip2 -dc [file.bz2] > [file]
copy
Test compressed file
$ pbzip2 -t [file.bz2]
copy

SYNOPSIS

pbzip2 [-d] [-k] [-p n] [-1..9] [options] [files]

DESCRIPTION

pbzip2 is a parallel implementation of bzip2 compression. It uses multiple CPU cores to compress and decompress files faster than single-threaded bzip2.
The tool produces files compatible with standard bzip2. Output can be decompressed by bzip2, bunzip2, or pbzip2 itself.
Parallelism scales with available processors. By default, all cores are used. The -p flag limits processor usage for background compression.
Compression levels work like bzip2: higher numbers produce smaller files but take longer. The parallel implementation helps offset slower compression speeds.
Memory usage scales with thread count. Each thread needs memory for its compression buffer. Limits can prevent memory exhaustion on large jobs.
Pipe mode supports streaming compression. Combined with tar, it enables parallel compressed archives.

PARAMETERS

-d

Decompress.
-z
Compress (default).
-k
Keep original files.
-p N
Number of processors to use.
-1 to -9
Compression level (9 = best).
-c
Output to stdout.
-f
Force overwrite.
-t
Test integrity.
-q
Quiet mode.
-v
Verbose mode.
-m N
Memory limit per thread (MB).
-r
Read entire file into RAM.

CAVEATS

Memory usage higher than bzip2. Best speedup on multi-core systems. Decompression speedup depends on how file was created.

HISTORY

pbzip2 was created by Jeff Gilchrist around 2005 to leverage multi-core processors for bzip2 compression. As multi-core CPUs became common, parallel compression tools like pbzip2 became essential for handling large data.

SEE ALSO

bzip2(1), pigz(1), lbzip2(1), xz(1)

> TERMINAL_GEAR

Curated for the Linux community

Copied to clipboard

> TERMINAL_GEAR

Curated for the Linux community