aws-kafka

Create a new MSK cluster

$ aws kafka create-cluster --cluster-name [cluster_name] --broker-node-group-info instanceType=[instance_type],clientSubnets=[subnet_id1 subnet_id2 ...] --kafka-version [version] --number-of-broker-nodes [number]

$ aws kafka describe-cluster --cluster-arn [cluster_arn]

$ aws kafka list-clusters

$ aws kafka create-configuration --name [configuration_name] --server-properties file://[path/to/configuration_file.txt]

$ aws kafka describe-configuration --arn [configuration_arn]

$ aws kafka list-configurations

$ aws kafka update-cluster-configuration --cluster-arn [cluster_arn] --configuration-info arn=[configuration_arn],revision=[configuration_revision]

$ aws kafka delete-cluster --cluster-arn [cluster_arn]

aws kafka subcommand [--options]

create-cluster
    Creates a new MSK cluster with specified settings like broker nodes, Kafka version, and storage

list-clusters
    Lists all MSK clusters in the current region, optionally filtered by tags or max results

describe-cluster
    Retrieves detailed information about a specific cluster using its ARN

delete-cluster
    Deletes an MSK cluster; cluster must be inactive

update-cluster-kafka-version
    Updates the Kafka version of an existing cluster

create-configuration
    Creates a custom Kafka configuration for clusters

list-configurations
    Lists all Kafka configurations in the account

get-bootstrap-brokers
    Returns bootstrap broker endpoints for connecting Kafka clients

reboot-cluster-nodes
    Reboots specified broker nodes in a cluster

update-monitoring
    Enables or updates CloudWatch monitoring for clusters

list-tags-for-resource
    Lists tags attached to an MSK resource

tag-resource
    Adds or overwrites tags on an MSK resource

--region
    AWS region (e.g., us-east-1); overrides default

--output
    Output format: json|text|table

--cli-input-json
    Reads parameters from JSON file

The aws kafka command is a subcommand of the AWS Command Line Interface (CLI) for interacting with Amazon Managed Streaming for Apache Kafka (MSK). Amazon MSK is a fully managed service that simplifies the deployment, operation, and scaling of Apache Kafka clusters in the AWS Cloud.

Using aws kafka, users can create, list, describe, update, and delete MSK clusters, configurations, VPC connections, and related resources. It supports operations like rebooting nodes, updating Kafka versions, managing monitoring, security, storage, and tagging. This command is essential for DevOps workflows, CI/CD pipelines, and automation scripts handling streaming data applications.

Key features include retrieving bootstrap brokers for client connections, listing cluster operations, and handling client authentication certificates. All actions require proper AWS credentials, IAM permissions (e.g., AmazonMSKFullAccess policy), and regional availability checks. The command integrates seamlessly with other AWS services like VPC, CloudWatch, and IAM.

For production use, it's recommended to use IAM roles over access keys and enable encryption, monitoring, and logging. Output formats include JSON, table, or text via --output option.

Requires AWS CLI v2 installed and configured with credentials. IAM permissions needed for each operation. Not all Kafka features supported (e.g., custom plugins limited). Regional service; check MSK availability. Deleting clusters is irreversible and incurs no charges post-deletion.

1. aws kafka create-cluster ...
2. aws kafka get-bootstrap-brokers ...
3. Connect clients using brokers

Attach policies like AmazonMSKFullAccess or custom IAM roles. Use least-privilege principle.

Introduced in AWS CLI v1.16.197 (2018) alongside Amazon MSK launch. Enhanced with multi-VPC support (2020), Kafka 3.x versions (2022), and storage updates (2023). Usage grew with streaming data adoption in microservices and real-time analytics.

aws(1), kcat(1), kafka-topics(1)