aws-kafka

Create a new MSK cluster

$ aws kafka create-cluster --cluster-name [my-cluster] --kafka-version [3.5.1] --number-of-broker-nodes 3 --broker-node-group-info file://[broker-config.json]

$ aws kafka list-clusters

$ aws kafka describe-cluster --cluster-arn [arn:aws:kafka:region:account:cluster/name/id]

$ aws kafka get-bootstrap-brokers --cluster-arn [arn:aws:kafka:region:account:cluster/name/id]

$ aws kafka update-broker-count --cluster-arn [arn:aws:kafka:region:account:cluster/name/id] --current-version [K1234] --target-number-of-broker-nodes 6

$ aws kafka get-compatible-kafka-versions --cluster-arn [arn:aws:kafka:region:account:cluster/name/id]

$ aws kafka delete-cluster --cluster-arn [arn:aws:kafka:region:account:cluster/name/id]

aws kafka command [options]

aws kafka is the AWS CLI interface for Amazon Managed Streaming for Apache Kafka (MSK), a fully managed service for running Apache Kafka. MSK handles cluster provisioning, configuration, patching, and monitoring while maintaining compatibility with open-source Kafka.
MSK supports both provisioned and serverless deployment modes, SASL/SCRAM and IAM authentication, encryption at rest and in transit, and integration with AWS services like Lambda, Kinesis Data Firehose, and Glue.

create-cluster

Create a new MSK cluster

Delete an MSK cluster

Get cluster configuration and status

List all clusters in the account

Get broker endpoints for client connections

Scale the number of brokers

Increase broker storage capacity

Upgrade Kafka version

Create a custom broker configuration

Restart a specific broker

Broker count can only be increased, not decreased. Storage can only be increased. Kafka version upgrades are one-way and cannot be rolled back. Cluster creation takes 15-20 minutes. MSK Serverless has different quotas than provisioned clusters.

Amazon MSK launched in December 2018 as a managed Kafka service. MSK Connect for managed connectors was added in 2021, and MSK Serverless launched in 2022 for automatic scaling without capacity planning. Tiered storage for cost optimization came in 2023.

aws(1), aws-kinesis(1), kafka-topics(1)