LinuxCommandLibrary

kcat

Consume and produce Kafka messages

TLDR

Consume messages starting with the newest offset

$ kcat -C -t [topic] -b [brokers]
copy

Consume messages starting with the oldest offset and exit after the last message is received
$ kcat -C -t [topic] -b [brokers] -o beginning -e
copy

Consume messages as a Kafka consumer group
$ kcat -G [group_id] [topic] -b [brokers]
copy

Publish message by reading from stdin
$ echo [message] | kcat -P -t [topic] -b [brokers]
copy

Publish messages by reading from a file
$ kcat -P -t [topic] -b [brokers] [path/to/file]
copy

List metadata for all topics and brokers
$ kcat -L -b [brokers]
copy

List metadata for a specific topic
$ kcat -L -t [topic] -b [brokers]
copy

Get offset for a topic/partition for a specific point in time
$ kcat -Q -t [topic]:[partition]:[unix_timestamp] -b [brokers]
copy

SYNOPSIS

kcat [options]

PARAMETERS

-h, --help
    Display help message and exit.

-s, --suppress-timestamp
    Suppress timestamp output.

-f, --follow
    Keep running and print new messages as they arrive.

-n, --no-decode
    Don't attempt to decode facility/severity numbers.

DESCRIPTION

kcat is a command-line utility for printing kernel logs. It monitors and displays messages generated by the Linux kernel, allowing users to observe system-level events, debugging information, and hardware-related activity in real-time. Unlike traditional methods like `dmesg`, kcat actively streams new kernel messages as they occur. It provides a simple interface to filter and present the kernel's output, aiding in identifying issues, tracking system behavior, and understanding the kernel's internal workings.

It's extremely useful for troubleshooting kernel-level problems, understanding hardware interactions, and monitoring system events that may not be visible in user-space logs. The command simplifies the process of continuously observing and capturing kernel output for analysis.

CAVEATS

kcat requires appropriate permissions to access kernel logs, typically root or membership in the `adm` group. The output can be verbose, so filtering or redirecting output may be necessary for effective analysis.

The program will exit when the connection to the kernel log socket is interrupted or when the end of input is reached, unless `-f` option is specified.

KERNEL LOGS

Kernel logs are a valuable resource for diagnosing system issues. They contain messages about hardware initialization, device driver activity, and other low-level events. Understanding and analyzing these logs is crucial for system administrators and developers to troubleshoot problems and optimize system performance.

TROUBLESHOOTING

When troubleshooting, use kcat to observe the system's behavior during the problematic operation. Look for error messages, warnings, or unexpected events in the kernel logs that might indicate the root cause of the issue.

HISTORY

kcat is a relatively recent utility, designed to provide a more convenient way to stream and monitor kernel logs compared to alternatives like `dmesg -w` or `tail -f /var/log/kern.log`. It aimed to provide a more streamlined interface specifically for interacting with the kernel log stream. Its development focuses on real-time kernel log viewing and simplicity, addressing specific shortcomings in existing log monitoring tools.

SEE ALSO

dmesg(1), journalctl(1)

Copied to clipboard