kcat

Consume messages starting with the newest offset

$ kcat -C -t [topic] -b [brokers]

$ kcat -C -t [topic] -b [brokers] -o beginning -e

$ kcat -G [group_id] [topic] -b [brokers]

$ echo [message] | kcat -P -t [topic] -b [brokers]

$ kcat -P -t [topic] -b [brokers] [path/to/file]

$ kcat -L -b [brokers]

$ kcat -L -t [topic] -b [brokers]

$ kcat -Q -t [topic]:[partition]:[unix_timestamp] -b [brokers]

kcat [ -C | -P ] [ -b brokers ] [ -t topic ] [ options ]

-b, --broker-list <brokers>
    Bootstrap broker(s) (host:port), comma-separated. Required unless configured otherwise.

-t, --topic <topic>
    Topic to consume/produce. Supports regex with -T.

-C, --consumer
    Consumer mode (default without -P).

-P, --producer
    Producer mode.

-p, --partition <part>
    Single partition to use (-1 for random).

-o, --offset <offset>
    Consumer start offset: beginning, end, tail, +/-N, or timestamp.

-c, --count <N>
    Produce/consume N messages then exit (0=unlimited).

-e, --consumer-exit-eof
    Consumer: exit on EOF (all partitions caught up).

-L, --list
    List brokers and topics (metadata dump).

-D, --dump-messages
    Dump full message details (headers, keys, etc.).

-f, --format <format>
    Output format string (printf-style, %s=msg, %k=key, etc.).

-F, --config-file <file>
    Load librdkafka config from file.

-X <prop>=<value>
    Set librdkafka property (e.g., -X sasl.mechanism=PLAIN).

-T
    Consumer: topic regex matching.

-Q
    Admin: query cluster metadata.

-S, --schema-registry <url>
    Schema Registry URL for Avro/etc.

-u, --unbuffered
    Disable buffering for real-time output.

kcat (formerly kafkacat) is a free, open-source command-line non-JVM Kafka client for Apache Kafka. It supports producing messages to topics, consuming from topics with flexible offset control, listing brokers/topics, metadata inspection, transactional APIs, and administrative operations like describe-config.

Key features include high performance via the librdkafka C library, support for Avro/Protobuf/JSON Schema serialization, SASL/SSL authentication, regex topic matching, message headers, exactly-once semantics, and compatibility with Kafka 0.8+ brokers. It reads from stdin or files for production and outputs to stdout with customizable formatting.

Unlike Kafka's console tools, kcat is lightweight, highly configurable via librdkafka properties, and ideal for scripting, testing, debugging, and benchmarking Kafka clusters. No JVM required, making it suitable for resource-constrained environments.

Requires librdkafka library. Not an official Apache Kafka tool; behavior depends on librdkafka version. High-throughput use needs tuned config (e.g., batching). No built-in offset commit in non-interactive mode unless specified.

Consume: kcat -b localhost:9092 -t mytopic
Produce from stdin: echo 'hello' | kcat -b localhost:9092 -P -t mytopic
List topics: kcat -b localhost:9092 -L