kafka

Start Kafka server

$ kafka-server-start.sh [config/server.properties]

$ kafka-server-stop.sh

$ kafka-topics.sh --create --topic [mytopic] --bootstrap-server [localhost:9092]

$ kafka-topics.sh --list --bootstrap-server [localhost:9092]

$ kafka-console-producer.sh --topic [mytopic] --bootstrap-server [localhost:9092]

$ kafka-console-consumer.sh --topic [mytopic] --from-beginning --bootstrap-server [localhost:9092]

kafka-server-start.sh config

Apache Kafka is a distributed event streaming platform. It provides high-throughput, low-latency message handling for real-time data pipelines and streaming applications.
Kafka organizes messages into topics, with partitions for parallelism and replication for fault tolerance. Producers send messages; consumers read them.

- Topic: Category for messages
- Partition: Ordered, immutable sequence
- Producer: Sends messages to topics
- Consumer: Reads messages from topics
- Broker: Kafka server node
- Consumer Group: Coordinated consumers

Requires Zookeeper (or KRaft in newer versions). Memory and disk intensive. Topic configuration affects retention. Rebalancing can cause delays.

Kafka was developed at LinkedIn and open-sourced in 2011. Named after author Franz Kafka, it became an Apache project and is now fundamental infrastructure for event-driven architectures.

zookeeper(1), kafka-topics(1), kafkacat(1)