ffmpeg-bitstream-filters

ffmpeg [input_options] -i [output_options]
-bsf[:stream_specifier] [=arguments]

-bsf[:stream_specifier] [=arguments]
    Applies a bitstream filter to the specified stream. stream_specifier can be 'v' for video, 'a' for audio, or a stream index (e.g., '0:v:0'). filter_name is the name of the bitstream filter to apply, and arguments are filter-specific options.

h264_mp4toannexb
    Converts H.264 bitstreams from MP4 (AVCC) format to Annex B format, which is typically required for streaming protocols like MPEG-TS or RTMP. It adds Start Code Prefixes (0x00000001) before NAL units and removes the NAL unit length fields.

aac_adtstoasc
    Converts AAC bitstreams from ADTS (Audio Data Transport Stream) format to ASC (AudioSpecificConfig) format. ADTS is common in streaming, while ASC is used in containers like MP4 to provide audio configuration only once.

remove_extra
    Removes 'extra' data from the bitstream. This might include padding bytes or metadata that is not essential for playback.

dump_extra
    Similar to remove_extra but instead of removing, it prints the extra data to stderr, useful for debugging.

chomp
    Removes trailing bytes from the stream, often used to clean up streams that might have garbage data at the end of packets.

noise
    Adds random noise to the bitstream. Primarily used for testing error resilience of decoders.

filter_units
    Filters specific NAL units (for H.264/H.265) or other unit types, allowing inclusion or exclusion based on type. For example, 'filter_units=remove_sps,remove_pps'.

Bitstream filters in FFmpeg are powerful tools used to manipulate the raw elementary stream data before it's decoded or after it's encoded, but before it's multiplexed into a container. Unlike video (-vf) or audio (-af) filters that operate on decoded (uncompressed) frames, bitstream filters work directly on the compressed bitstream. This makes them ideal for tasks such as modifying stream headers, fixing common containerization issues, or adding/removing specific data elements that are part of the elementary stream. Common use cases include converting H.264 streams from MP4-style (AVCC) to Annex B format for streaming protocols, converting AAC streams from ADTS to ASC (AudioSpecificConfig) format, or stripping padding bytes. They are specified using the -bsf (or -bsf:v, -bsf:a) option followed by the filter name and optional arguments.

Bitstream filters operate on the raw compressed data. Incorrect usage can lead to corrupted streams that are unplayable or cause decoders to crash. They do not re-encode the stream, so they are fast but limited to structural modifications of the elementary stream. Always test changes thoroughly. Not all containers or codecs are compatible with every bitstream filter.

It's crucial to distinguish between bitstream filters (-bsf) and stream filters (-vf for video, -af for audio). Bitstream filters work on the raw, compressed data bytes of the elementary stream, modifying its structure (e.g., adding start codes, removing headers). Stream filters, conversely, operate on the decoded, uncompressed frames or samples, allowing for operations like scaling, cropping, deinterlacing, or applying audio effects. Bitstream filters are applied before decoding or after encoding, while stream filters are applied within the decoding/encoding pipeline.

When using the -bsf option, it's often beneficial to specify which stream the filter should apply to. For example, -bsf:v h264_mp4toannexb applies the filter only to the video stream. Similarly, -bsf:a aac_adtstoasc applies it to the audio stream. You can also target specific streams using indices like -bsf:0:v h264_mp4toannexb for the first video stream from the first input.

FFmpeg has evolved as a comprehensive multimedia framework since its inception. Bitstream filters were integrated into its core functionality to address the complexities of various codec formats and container specifications. As multimedia standards progressed and new codecs emerged, the need for direct manipulation of elementary streams became apparent, leading to the development and expansion of the '-bsf' option and its associated filter library. This allows users to adapt streams for different playback environments or conform to specific protocol requirements without undergoing a full decode/encode cycle.

ffmpeg(1)