cephadm

Bootstrap a new Ceph cluster on the current host

Add a new host to the cluster
Deploy a specific service (e.g., mgr, mon, osd)
Check the status of cluster services
Enter a shell environment inside the Ceph container
Remove a service from the cluster

cephadm [global-options] <command> [command-options] [arguments]

Common commands:
cephadm bootstrap --mon-ip <IP>
cephadm add-host --ssh-user <user> --ssh-private-key <path_to_key> <hostname>
cephadm shell [--container-image <image>]
cephadm ls
cephadm rm <daemon_name>
cephadm deploy --fsid <FSID> --config <path> --keyring <path> [daemon_type] [daemon_id]
cephadm upgrade --ceph-version <version>

--fsid <id>
    Specifies the cluster FSID. Required for some commands if not already known.

--config <path>
    Path to the Ceph configuration file.

--keyring <path>
    Path to the Ceph keyring file.

--ssh-user <user>
    The SSH user for connecting to remote hosts.

--ssh-private-key <path>
    Path to the SSH private key file.

--skip-ssh-config
    Do not generate an SSH config file for hosts.

--image <image>
    Specify the container image to use (e.g., quay.io/ceph/ceph:v18.2.0).

--docker
    Use Docker as the container runtime (default is Podman).

--podman
    Use Podman as the container runtime (default).

--log-file <path>
    Path to save cephadm logs.

--verbose, -v
    Enable verbose output for debugging.

--dry-run
    Show what would be done without actually making changes.

--force
    Bypass certain safety checks.

cephadm is the official deployment and management tool for Ceph storage clusters, introduced in Ceph Octopus. It streamlines the entire lifecycle of a Ceph cluster, from initial bootstrap and deployment to daily operations, upgrades, and scaling.

It operates by connecting to hosts via SSH, deploying and managing Ceph daemons as systemd units running inside containers (either Podman or Docker). This approach offers several benefits:

• Simplicity: A single tool handles all aspects of cluster management.
• Consistency: Ensures all daemons are deployed uniformly across hosts.
• Isolation: Containerization prevents dependency conflicts with the host system.
• Robustness: Leverages systemd for daemon supervision and restarts.

cephadm communicates with the Ceph orchestrator module within the Ceph Manager daemon (MGR) to understand the desired state of the cluster and enforce it. It enables declarative management, allowing users to define the cluster's desired configuration, and cephadm works to achieve that state. This tool is fundamental for deploying production-grade Ceph clusters on bare metal or virtual machines.

• SSH Connectivity Required: cephadm heavily relies on passwordless SSH access to all hosts in the cluster. SSH keys must be correctly configured.
• Container Runtime Dependency: Requires either Podman (default and recommended) or Docker to be installed and operational on all target hosts.
• Network Configuration: Proper network configuration, including dedicated public and cluster networks (for performance and isolation), is crucial for production deployments.
• Complexity for Beginners: While simplifying many aspects, cephadm still requires a fundamental understanding of Ceph architecture and Linux system administration. Debugging can be challenging without this knowledge.
• System Requirements: Target hosts must have systemd and a compatible kernel version.
• Idempotency: While largely idempotent, some operations might require manual intervention if issues arise.

cephadm's typical workflow begins with a bootstrap command on a single host, which sets up the initial monitor (MON), manager (MGR), and an cephadm management key. This host becomes the entry point for managing the entire cluster.

Subsequent hosts are added using add-host, and then cephadm interacts with the orchestrator module running in the MGR to deploy additional daemons (OSDs, MDS, RGW, etc.) based on declarative specifications (e.g., ceph orch apply osd --all-available-devices). This allows for dynamic scaling and self-healing capabilities.

cephadm relies on official Ceph container images, which contain all necessary Ceph binaries and dependencies. This ensures consistency and simplifies upgrades. The upgrade command orchestrates the rolling upgrade of all daemons in the cluster, pulling new container images and restarting services systematically to minimize downtime.

cephadm emerged as the flagship deployment tool for Ceph starting with the Ceph Octopus release (2020). It was developed to address the limitations and complexities of previous deployment methods, such as ceph-deploy, which often required manual intervention and lacked a unified lifecycle management approach.

Its introduction marked a significant shift towards a more robust, declarative, and container-native way of managing Ceph clusters. By leveraging systemd and container technologies (primarily Podman), cephadm aimed to provide a simpler, more consistent, and reliable experience for deploying and operating Ceph on bare metal and virtual machines, consolidating various tasks into a single, cohesive tool. It has since become the recommended and primary method for new Ceph deployments.

ceph(8), systemctl(1), ssh(1), podman(1), docker(1), ceph-volume(8), ceph-mgr(8)