LinuxCommandLibrary

kubectl-debug

Debug running Kubernetes pods

TLDR

Create an interactive debugging session in a pod and immediately attach to it

$ kubectl debug [pod_name] [[-it|--stdin --tty]] --image busybox
copy

Create a debug container with a custom image and name
$ kubectl debug --image [image] [[-c|--container]] [container_name] [pod_name]
copy

Create an interactive debugging session on a node and immediately attach to it (the container will run in the host namespaces and the host's filesystem will be mounted at /host)
$ kubectl debug node/[node_name] [[-it|--stdin --tty]] --image busybox
copy

Create a copy of a pod and add a debug container to it
$ kubectl debug [pod_name] [[-it|--stdin --tty]] --image [image] --copy-to [pod_copy_name]
copy

Create a copy of a pod and change the command of a specific container
$ kubectl debug [pod_name] [[-it|--stdin --tty]] --copy-to [pod_copy_name] --container [container_name] -- [command]
copy

Create a copy of a pod and change the image of a specific container
$ kubectl debug [pod_name] --copy-to [pod_copy_name] --set-image [container_name]=[image]
copy

Create a copy of a pod and change all container images
$ kubectl debug [pod_name] --copy-to [pod_copy_name] --set-image '*=[image]'
copy

Create an ephemeral debug container and target a specific container (useful for debugging distroless containers)
$ kubectl debug [pod_name] [[-it|--stdin --tty]] --image [image] --target [target_container_name]
copy

SYNOPSIS

kubectl debug [options] (POD | TYPE/NAME) [flags] [-- COMMAND [args...]]

PARAMETERS

-c, --container
    Specify the container within the pod to debug. If omitted, the first container is chosen.

-i, --image
    Set the container image to use for the debug container. Defaults vary (e.g., busybox, nicolaka/netshoot).

--profile
    Use a predefined debug profile which sets a specific image and pre-installs common debugging tools (e.g., netshoot, golang).

--ephemeral
    Use an ephemeral container for debugging. Requires Kubernetes 1.25+ for stable support.

--target
    Specify the target container whose process namespace will be shared with the debug container (typically used with --ephemeral).

--share-process-namespace
    Enable sharing of the process namespace with other containers in the pod, allowing visibility into their processes.

-p, --port-forward
    Automatically set up port forwarding from your local machine to the debug container.

--attach
    Attach to the debug container immediately after creation (default behavior).

--shell
    Specify the shell command to run in the debug container (e.g., bash). Defaults to sh -l or bash -l.

--tty
    Allocate a pseudo-TTY for the debug container (default is true).

--stdin
    Pass stdin to the container (default is true).

--wait
    Wait for the debug container to be ready before attaching.

DESCRIPTION

The kubectl-debug plugin allows users to attach a new, temporary container to an existing running pod for in-depth debugging. This enables the use of specialized debugging tools (like strace, tcpdump, gdb) directly within the pod's environment without modifying the original container images or restarting the application.

It can operate by creating either an ephemeral container (for Kubernetes 1.25+ clusters, mirroring the native kubectl debug command's functionality) or a regular sidecar container within the pod for older Kubernetes versions. By often sharing the process namespace, the debug container can observe and interact with processes running in the target container, providing powerful insights into application behavior and system calls.

CAVEATS

Requires appropriate Kubernetes RBAC permissions to create debug or ephemeral containers. Full functionality, especially with ephemeral containers and process namespace sharing, depends on the Kubernetes cluster version (1.25+ for stable ephemeral containers). The chosen debug image must contain the necessary debugging utilities.

EPHEMERAL CONTAINERS VS. SIDECARS

This plugin can create either an ephemeral container or a regular sidecar container. Ephemeral containers are temporary, non-persisted containers ideal for debugging, which are not part of the pod's permanent specification. Sidecars, while effective, become part of the pod's definition and may require manual cleanup.

KREW PLUGIN

kubectl-debug is widely available and easily installed via Krew, the package manager for Kubernetes CLI plugins. This simplifies its deployment and maintenance for users.

HISTORY

The kubectl-debug plugin was developed to address limitations in debugging running Kubernetes pods, particularly before the native kubectl debug command with ephemeral container support became mature (stable in Kubernetes 1.25). It provided a flexible way to inject a temporary, feature-rich debugging environment into a pod without modifying its definition or restarting it, enabling deeper analysis than simple kubectl exec could offer.

SEE ALSO

kubectl exec(1), kubectl attach(1), kubectl logs(1), kubectl debug(1) (native command), strace(1), tcpdump(8)

Copied to clipboard