An ephemeral container is a temporary container that you add to a Pod that is already running, so you can inspect it without rebuilding the image or restarting the application. It exists for troubleshooting only. You add it with kubectl debug, it does its job, and it stays in the Pod spec as a finished container until the Pod goes away. This guide covers how it works, the commands you will use, the limits you will hit and the security questions to settle first.
TL;DR
- An ephemeral container is added to a running Pod for user-initiated tasks such as debugging. It cannot be defined when the Pod is created.
kubectl debug -it <pod-name> --image=<image-name>adds one and attaches to its console right away.- It helps most with distroless images that have no shell, and with Pods where
kubectl execis not enough. - It has no resource guarantees, never restarts, and cannot be changed or removed once added.
- Ephemeral containers run with the privileges of a normal container, so guard the
/ephemeralcontainerspath in your admission policy.
Table of Contents
- What an ephemeral container is
- How it differs from a regular container
- Add an ephemeral container with kubectl debug
- Debugging distroless images and shared process namespaces
- Limits you will run into
- Security: what administrators should settle first
- Ephemeral containers are not ephemeral volumes
- Our view: debugging should not need five tools
- Where Kubezilla Desktop fits
- A practical checklist for your next debug session
- FAQ
- Sources
What an ephemeral container is
A Pod is meant to be disposable. When something misbehaves, the usual fix is to delete the Pod and let the controller create a new one. That fix destroys the evidence. You lose the process state, the open connections and the odd condition you were trying to catch.
According to the Kubernetes documentation, an ephemeral container is a special type of container that runs temporarily in an existing Pod to accomplish a user-initiated action such as troubleshooting. You bring your own tools in a separate image, and the application image stays untouched.
That matters because production images are often stripped down on purpose. They carry no shell, no package manager, no curl, no strace. Those omissions are good for security and image size, and they make live debugging painful. An ephemeral container lets you attach a toolbox to the Pod without putting the tools in the application image.
The design also moved debugging into the platform itself. The Google Open Source Blog describes the feature as a first-class debugging experience that replaces client-side hacks such as copying binaries into a running container.
How it differs from a regular container
Ephemeral containers use the same ContainerSpec as regular containers, but the API disallows several fields. Here is the practical comparison.
| Aspect | Regular container | Ephemeral container |
|---|---|---|
| When it is defined | At Pod creation | After the Pod is running |
| Restart on exit | Follows the Pod's restart policy | Never restarted automatically |
| Resource requests and limits | Supported | Not supported |
| Ports, liveness and readiness probes | Supported | Disallowed |
| Change or remove after adding | Through a new Pod template | Not possible |
| Static Pods | Supported | Not supported |
| Purpose | Run the workload | Troubleshooting and inspection |
The first column of rules comes from the Pod documentation on ephemeral containers. The enhancement proposal behind the feature adds the reason: ephemeral containers are not meant for building services, so the API rejects service-building fields such as ports.
The missing resource guarantees deserve a second look. Your debug container shares the Pod's resources but reserves none of its own. A heavy tool run inside it competes with the application you are trying to diagnose. Keep debug sessions short and light.
Add an ephemeral container with kubectl debug
You rarely write the API object by hand. The kubectl debug command does it for you. The kubectl reference gives the basic form:
kubectl debug -it <pod-name> --image=<image-name>
This adds the debugging container to the Pod and attaches to its console immediately. A typical session looks like this:
kubectl debug -it api-7d9f6c5b8-x2k4p --image=busybox:1.36 --target=api
The --target flag names the container whose process namespace you want to join. Without it, your debug container may not see the application's processes. Whether it works depends on your container runtime, so test it on a non-critical Pod before you need it during an incident.
A few habits make this smoother:
- Pick the image for the task. A small image with a shell is enough for file and process checks. Use an image with network tools when the question is DNS or connectivity.
- Name the target container. In a multi-container Pod, say which container you are inspecting.
- Choose a profile deliberately. More on that below.
- Exit cleanly. The container stops when your shell exits, but its record stays in the Pod spec.
Debugging profiles
The command offers profiles named general, baseline, restricted, netadmin and sysadmin. According to the kubectl documentation, these set the security context and permissions of the debugging session. Choose the narrowest one that answers your question. A network capture needs netadmin, while a quick look at files rarely needs anything beyond baseline or restricted.
The same page warns that some capabilities granted by a profile may not work when the target Pod runs as a non-root user. If a tool fails with a permission error in a hardened Pod, that is the first thing to check.
Since Kubernetes 1.31 you can also supply a custom profile. The Kubernetes blog post on custom profiling explains that fields such as command, image, lifecycle, volume devices and container name are deliberately not customizable, to keep the feature usable.
Debugging distroless images and shared process namespaces
This is where ephemeral containers earn their place. The Kubernetes docs say they are useful for troubleshooting distroless images that lack a shell or debugging utilities, and for cases where kubectl exec is insufficient.
Consider an application built on a distroless base. kubectl exec fails because there is no shell binary to launch. You could rebuild the image with a shell, but that changes the thing you are trying to observe, and the bug may not return.
With an ephemeral container you do this instead:
- Run
kubectl debug -it <pod-name> --image=busybox:1.36 --target=<app-container>. - List processes and confirm the application is running as expected.
- Check environment, network reachability and DNS from inside the Pod's network namespace.
- Exit, and leave the application undisturbed.
Process namespace sharing is what makes step 2 possible. The Pod documentation notes that when it is enabled, you can see processes from other containers in the Pod. Without it, your debug container sees only its own processes, and a ps command will look suspiciously empty.
When you cannot touch the live Pod
Sometimes you do not want to attach to the running Pod at all, for example when the debug tool might disturb it. kubectl debug also supports creating a copy of the Pod with extra tooling added through its --copy-to option, described in the kubectl reference. The copy is a different Pod, so it will not reproduce a problem that depends on the original's state, but it is a safe place to try heavier tools.
Limits you will run into
Know these before an incident, not during one. The Kubernetes documentation states that ephemeral containers:
- do not support resource requests or limits;
- are not restarted when they exit;
- cannot be changed or removed once added;
- are not supported by static Pods.
There are practical consequences. If your debug shell crashes, you start a new ephemeral container, and the old entry stays in the Pod's status. A Pod that has collected several of them looks cluttered, though it is harmless. Pods are replaced on rollout anyway, so the clutter disappears with the next deployment.
The design proposal also records a minor race condition. When you create an ephemeral container and attach immediately, the container may begin executing before your client connects, so you can miss the first lines of output. If your tool prints something important at startup, start with a shell and run the tool by hand.
Finally, ephemeral containers are for diagnosis. Do not use them to patch a production problem. A change made in a debug shell lives only in that Pod and vanishes at the next restart. Fix the cause in the manifest or the image. If the fix is a restart, our guide to restarting Pods in Kubernetes walks through the safe methods.
Security: what administrators should settle first
An ephemeral container is powerful because it can reach into a running workload. That is also the risk.
The design proposal states that ephemeral containers function with the same privileges as a standard container. They do not get a lighter security model just because they are temporary. It also says that admission plugins must be updated to guard the /ephemeralcontainers path so policy is enforced consistently. If your admission controls check Pod creation and updates but ignore that subresource, a debug container could slip around them.
A sensible baseline for a team:
- Treat the permission as sensitive. Grant access to the
ephemeralcontainerssubresource through RBAC only to people who already have a reason to enter production Pods. - Check admission policy. Confirm that your policy engine evaluates the subresource, and test it with a deliberately disallowed image.
- Use the least powerful profile. Reach for sysadmin only when a lower profile cannot answer the question.
- Keep an audit trail. Debug sessions touch live workloads, so you want to know who did what and when.
- Remember the data. A debug container in a Pod can reach whatever that Pod can reach, including mounted secrets.
The same discipline applies to the machine you debug from. A kubeconfig with production access should not sit in a synced folder or a browser extension. Our guide to kubeconfig context management covers switching and merging contexts without ending up on the wrong cluster.
Ephemeral containers are not ephemeral volumes
The search results for this keyword mix up two unrelated ideas, so here is the distinction. Ephemeral containers are about running a temporary process inside a Pod. Ephemeral volumes are about storage. According to the Kubernetes storage documentation, ephemeral volumes hold scratch data or injected data and follow the Pod's lifetime. Generic ephemeral volumes can be backed by any driver that supports dynamic provisioning, much like a PersistentVolumeClaim, but they are managed as part of the Pod's lifecycle.
If your question is whether data survives a Pod restart, you want the storage page. If your question is how to get a shell into a Pod that has none, you are in the right place.
Our view: debugging should not need five tools
We build Kubezilla Desktop, so we watch how engineers actually debug. The ephemeral container is a good feature that is awkward to use at the speed an incident demands.
A typical session involves a terminal to find the right context, a second to find the Pod name, a long kubectl debug command typed from memory, and often a browser tab for logs and metrics. Every handoff is a chance to type the wrong namespace or target the wrong cluster. The risky part of debugging is rarely the container image. It is being on the wrong context at 2 a.m.
Our position is that cluster access should be local-first. Your kubeconfigs belong on your machine, stored securely, with no cloud sync and no telemetry. Reaching a cluster behind a bastion should be an SSH tunnel you can see, not a VPN you have to fight. And the tool should be light enough that it does not compete with your editor and browser for memory, which is why we built it in Rust.
We also think a debug tool should make the dangerous thing obvious. A pod shell, live logs, metrics and a YAML view in one window mean you can confirm what is wrong before you inject anything into a live Pod, and sometimes you find the answer without needing a debug container at all.
— Margus
Where Kubezilla Desktop fits
Kubezilla Desktop is a native app for Linux, macOS and Windows that brings pod terminals, live log streaming, metrics, YAML editing, workload scaling, Helm and ArgoCD operations and PVC file browsing into one interface. It connects to multiple clusters, supports SSH tunnels to remote clusters, and keeps kubeconfigs in secure local storage. It costs €5 per month.
It does not replace knowing kubectl debug. It shortens the path around it: pick the cluster, open the Pod, read the logs and metrics, and open a shell without juggling terminals. See the Kubernetes desktop app page for the full feature list, or the pricing page for plan details. If you mostly want faster shell access, read our piece on the Kubernetes pod shell. For clusters behind a jump host, see SSH tunnels for Kubernetes.
A practical checklist for your next debug session
- Confirm the context and namespace before you type anything.
- Read logs and recent events first. Many problems end here.
- If you need a shell and the image has none, run
kubectl debugwith a small image and--target. - Pick the lowest profile that works.
- Keep the session short, because the container has no resource guarantees.
- Write down what you found, then fix the manifest or image rather than the live Pod.
- Let the next rollout clear the old debug container.
When you are about to apply a fix, validate it first. Our article on Kubernetes dry runs explains why only the server-side option checks admission.
FAQ
What is an ephemeral container in Kubernetes?
It is a temporary container added to an already running Pod for a user-initiated task, usually debugging. It is not part of the original Pod definition, it is never restarted automatically, and it has no resource guarantees. The Kubernetes documentation describes it as a special container type for this purpose.
Can I remove an ephemeral container after I finish?
No. Once added, an ephemeral container cannot be changed or removed. It stops when its process exits, and its entry disappears only when the Pod itself is replaced. That is normal behavior, not a leak.
Do I need ephemeral containers if I can use kubectl exec?
Only when exec is not enough. If the container has a shell and the tools you need, exec is simpler. If the image is distroless or minimal, an ephemeral container supplies the missing tools without changing the application image.
Are ephemeral containers safe to use in production?
They are safe when access is controlled. They run with the privileges of a standard container, so restrict the RBAC permission, make sure admission policy covers the /ephemeralcontainers path, and choose the least powerful debug profile. Treat each session like any other privileged access to a production workload.
Are ephemeral containers the same as ephemeral storage?
No. Ephemeral containers are temporary debugging processes. Ephemeral volumes and ephemeral storage describe data that lives only as long as the Pod. They solve different problems.
Sources
- Ephemeral Containers, Kubernetes documentation
- KEP-277: Ephemeral Containers, Kubernetes Contributors
- Introducing Ephemeral Containers, Google Open Source Blog
- kubectl debug reference, Kubernetes
- Kubernetes 1.31: Custom Profiling in Kubectl Debug
- Ephemeral Volumes, Kubernetes documentation