Kubernetes Drain and Cordon: Commands, Flags, Uncordon

Cordon stops new pods from landing on a node. Drain does that and also evicts the pods already running there. Run kubectl cordon <node> when you only want to freeze scheduling, run kubectl drain <node> --ignore-daemonsets when the node must be emptied for maintenance, and run kubectl uncordon <node> when the work is done. The rest of this guide covers the order of operations, the flags that matter, the reasons a drain hangs, and how to confirm the node is healthy again.

Kubezilla Desktop

Manage Clusters From One Desktop App

Kubezilla Desktop brings multiple clusters, live logs, metrics, Helm, and ArgoCD operations into a unified native interface.

Explore Kubezilla Desktop

TL;DR

Cordon marks a node unschedulable. Existing pods keep running. Drain cordons the node, then evicts its pods so they reschedule elsewhere.
The standard maintenance sequence is: check PodDisruptionBudgets, drain with --ignore-daemonsets, do the work, then kubectl uncordon.
Drain stalls on three things: bare pods (need --force), emptyDir data (needs --delete-emptydir-data), and PodDisruptionBudgets that would be violated.
Uncordoning only reopens scheduling. It does not move pods back, so verify the node and rebalance if you need to.

What cordon and drain actually do
Before you drain: a short checklist
How to cordon a node
How to drain a node
Why a drain hangs, and what to do
Do the maintenance, then uncordon
What about node shutdowns and reboots?
Our perspective on drain and cordon
Run node maintenance from one window
A worked example
Cordon and drain in automation
FAQ
Sources

What cordon and drain actually do

The kubectl drain reference puts it plainly: cordoning marks a node unschedulable so no new pods arrive, while draining marks it unschedulable and also evicts or deletes the pods on it. Mirror pods (static pods the kubelet mirrors into the API) are the exception, since they cannot be removed through the API server.

The difference matters in practice:

	Cordon	Drain
New pods scheduled to node	No	No
Existing pods	Keep running	Evicted or deleted
Disruption to workloads	None immediately	Yes, pods restart elsewhere
Typical use	Hold a node while you investigate, or stage a rollout	Reboot, upgrade, resize, or retire a node
Reverse with	`kubectl uncordon`	`kubectl uncordon`

Think of cordon as closing the door and drain as asking everyone inside to leave. A cordoned node is a safe place to look at a problem, because nothing you are watching will be replaced or moved. A drained node is ready for the kernel patch, the kubelet upgrade, or the hardware swap.

What happens inside a drain

Drain first cordons the node. It then evicts pods one by one. According to the Kubernetes documentation on safely draining a node, kubectl drain safely evicts pods and lets their containers terminate gracefully, within the termination period each pod asks for. The Disruptions page adds that an evicted pod is terminated gracefully and honors the terminationGracePeriodSeconds in its PodSpec.

When the API server supports eviction, drain uses the Eviction API. Otherwise it falls back to ordinary deletion, as the drain reference explains. The difference is important: eviction respects PodDisruptionBudgets, plain deletion does not.

Pods owned by a Deployment, ReplicaSet, StatefulSet, or Job are recreated by their controller, and the scheduler places the replacements on other nodes. Because the node is cordoned, the replacements cannot come back to it.

Drain sequence and pod replacement across nodes

Before you drain: a short checklist

Most bad drains are bad because of what nobody checked first. Spend two minutes on this list.

Confirm capacity elsewhere. The remaining nodes must have enough CPU and memory for the evicted pods. If they do not, replacements sit in Pending.
Look at PodDisruptionBudgets. The Kubernetes docs say to configure PDBs first if availability matters. A PDB limits how many pods of a replicated application can be down at once during voluntary disruptions, and a drain is a voluntary disruption.
Count replicas. A Deployment with one replica and a PDB that requires one available pod will block the drain forever. Either add a replica or accept the downtime consciously.
Find unmanaged pods. Pods with no controller will not come back after eviction.
Find local data. Pods using emptyDir lose that data when evicted.
Note pinned workloads. Setting a pod's nodeName directly bypasses the scheduler, and the safe-drain documentation warns this can leave a pod running on a drained or cordoned node.

Useful commands for the survey:

kubectl get nodes
kubectl get pods -A -o wide --field-selector spec.nodeName=<node>
kubectl get pdb -A

If you juggle several clusters, this survey is where context mistakes happen. Confirm the active context before you run anything that evicts pods. A desktop client that shows the cluster name and node list side by side makes that check hard to skip.

How to cordon a node

kubectl cordon <node-name>

The node now shows Ready,SchedulingDisabled in kubectl get nodes. Existing pods are untouched. Under the hood the node's spec.unschedulable field is set to true, and the control plane adds the node.kubernetes.io/unschedulable:NoSchedule taint.

Good reasons to cordon without draining:

A node looks flaky and you want to stop new work landing on it while you read logs and metrics.
You are staging a rolling replacement: cordon all old nodes first, then drain them one at a time, so evicted pods never land on another old node.
You are preparing a node for a later maintenance window and want the load to shrink naturally as pods are rescheduled by normal rollouts.

How to drain a node

The baseline command from the safe-drain task page:

kubectl drain --ignore-daemonsets <node-name>

The --ignore-daemonsets flag is nearly always required. DaemonSet pods are skipped because the DaemonSet controller would only replace them immediately, and its pods tolerate the unschedulable taint. Without the flag, drain refuses to proceed when it finds them.

A fuller, still conservative version:

kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data --timeout=300s

Flags worth knowing

Flag	What it does	When to use it
`--ignore-daemonsets`	Skips DaemonSet-managed pods	Almost every drain
`--delete-emptydir-data`	Allows eviction of pods using `emptyDir` volumes	Only when that data is disposable
`--force`	Deletes pods with no managing controller	Only when you accept they will not return
`--grace-period=<seconds>`	Overrides each pod's termination grace period	Rarely; it overrides what the app asked for
`--timeout=<duration>`	Gives up if the drain does not finish	Scripts and runbooks, so nothing hangs forever
`--dry-run=client`	Shows what would be evicted without doing it	Before any drain you are unsure about

The flag behavior comes from the drain reference, which also states that drain will not delete non-mirror pods lacking a replication controller, ReplicaSet, DaemonSet, StatefulSet, or Job unless you pass --force.

If you want a general habit for previewing risky operations, our write-up on Kubernetes dry runs and which one checks admission explains the difference between client and server checks.

Draining several nodes

The documentation notes that multiple kubectl drain commands can run in parallel and Kubernetes will still respect configured PDBs. That is a safety net, not an invitation. Draining three nodes at once can leave too little capacity for replacements, so in a small cluster drain one node at a time and wait for pods to become Ready before starting the next.

Why a drain hangs, and what to do

A drain that sits there is usually doing exactly what it should. Work through the likely causes before reaching for --force.

A PodDisruptionBudget would be violated

When evicting a pod would breach a PDB, the eviction request is rejected and drain retries after a short backoff. It keeps retrying until the budget allows it or your --timeout expires. Check what is blocking:

kubectl get pdb -A
kubectl describe pdb <name> -n <namespace>

Look at Allowed disruptions. If it is 0, ask why. Often a replacement pod is still starting, or the application has too few replicas for its own budget. Fix the cause: scale up, wait for readiness, or have the application owner revisit the PDB. Deleting a PDB to get past a drain defeats the point of having one. For scaling commands, see our guide to scaling a Kubernetes deployment.

Pods with no controller

A bare pod created with kubectl run or a manifest has nothing to recreate it. Drain stops and tells you. Decide whether the pod matters. If it does, recreate it as a Deployment or Job first. If it is disposable, --force removes it.

Pods using emptyDir

Drain stops on pods with emptyDir volumes because their data disappears on eviction. If a cache or scratch directory is all that is in there, add --delete-emptydir-data. If something valuable lives there, copy it out first.

Slow termination

Pods that take a long time to shut down hold the drain open for their full grace period. Read the pod's terminationGracePeriodSeconds before cutting it with --grace-period, since shortening it can mean dropped requests or unflushed writes.

Managed platforms add their own limits. Google documents a 20-minute drain timeout before nodes enter maintenance mode on Google Distributed Cloud for bare metal (Kubernetes 1.29 and earlier versions in its docs). Oracle's container service documents a default eviction grace period of 60 minutes, which is also the maximum, in its guidance on cordoning and draining managed nodes. If you run on a managed service, read your provider's page so your maintenance window fits.

Replacement pods stuck in Pending

If evicted pods cannot be scheduled, the drain may finish but the application is degraded. Check events with kubectl get events -A --sort-by=.lastTimestamp and look at node resources. Our connectivity troubleshooting guide helps when pods run but cannot be reached.

Do the maintenance, then uncordon

With the node empty, patch, reboot, upgrade the kubelet, or resize it. When it reports Ready again, reopen scheduling:

kubectl uncordon <node-name>

This is the answer to the common question of how to add a drained node back: the safe-drain documentation states that kubectl uncordon makes the node schedulable again after maintenance.

Uncordon does not move anything. Pods that were evicted stay where they landed, and the freshly uncordoned node starts empty. New pods and future rollouts will use it. If you want load spread back, restart a Deployment so its replacements are scheduled across all nodes. Our guides on restarting a Kubernetes deployment and restarting pods safely cover how to do that without taking the service down.

Verify after uncordoning

kubectl get nodes shows Ready with no SchedulingDisabled.
kubectl describe node <node> shows no unexpected taints, and conditions such as MemoryPressure and DiskPressure are false.
DaemonSet pods (logging, networking, monitoring agents) are running on the node.
A new test workload, or the next rollout, schedules onto the node.
Application dashboards and error rates look normal.

What about node shutdowns and reboots?

Drain is a voluntary, operator-driven step. If a node shuts down without being drained, the kubelet's graceful node shutdown feature can still terminate pods in two phases: regular pods first, then critical pods, as described in the node shutdown documentation. That is a fallback for surprises. For planned work, drain first, because drain respects PodDisruptionBudgets and graceful shutdown on the node does not replace that coordination.

Our perspective on drain and cordon

We build Kubezilla Desktop for people who touch several clusters a day, and the pattern we see in node maintenance is rarely a lack of knowledge about the commands. It is context. Which cluster is this terminal pointed at? Which pods are on this node? Is that PDB already at zero allowed disruptions? Each answer lives in a different command, a different tab, or a different terminal window.

That is why we designed Kubezilla as a single native app where you connect to your clusters, see nodes, pods, logs, and metrics in one place, and keep kubeconfigs stored locally on your machine. Our approach is local-first: no cloud sync and no telemetry, so the credentials that can drain production nodes do not leave your laptop. Built in Rust, it is designed to stay light on RAM and start fast, which matters when you are in the middle of a maintenance window and do not want to wait on a heavy UI.

Our view on practice: cordon early, drain deliberately, and always watch pods come back on other nodes before you touch the node you emptied. Seeing live logs and metrics next to the node list turns that watching step from a chore into a habit. And if a cluster sits behind a bastion, the built-in SSH tunneling means the same view works there too, without a separate VPN session.

— Margus

Run node maintenance from one window

If you manage nodes across more than one cluster, Kubezilla Desktop gives you a unified interface for cluster and resource operations, live log streaming, metrics, pod terminals, YAML editing, and workload scaling, with Helm and ArgoCD actions close at hand. It runs on Linux, macOS, and Windows and costs €5 per month. See the details on the Kubezilla Desktop pricing page.

Kubezilla Desktop

For remote clusters, our guide to Kubernetes SSH tunnels and certificate errors shows how tunnels fit into this workflow, and the Kubernetes desktop app page explains how the client handles multiple clusters.

A worked example

Suppose a three-node cluster runs a web Deployment with three replicas, a PDB with minAvailable: 2, and a logging DaemonSet. You need to patch node-b.

kubectl get pods -A -o wide --field-selector spec.nodeName=node-b shows one web pod, the logging agent, and a cache pod with an emptyDir.
kubectl get pdb shows 1 allowed disruption, so evicting the single web pod on node-b is fine.
You run kubectl drain node-b --ignore-daemonsets --delete-emptydir-data --timeout=300s, because the cache is rebuildable.
The web pod is evicted and rescheduled to node-a or node-c. You watch it become Ready.
You patch and reboot node-b, then run kubectl uncordon node-b.
You confirm node-b is Ready, the logging agent is running on it, and the error rate stayed flat.

Now the same cluster with a PDB of minAvailable: 3. The drain would loop on that web pod indefinitely, because evicting it breaks the budget. The fix is not --force, which would not bypass a PDB anyway, but adding a fourth replica so there is room to move one.

Cordon and drain in automation

For fleets, people automate this. Upgrade tooling and node-pool managers cordon and drain for you, and the Disruptions page tells cluster managers to use tools that respect PDBs. Whatever you adopt, check three things: does it honor PDBs, does it have a timeout, and does it drain one node at a time? If the answer to any is no, expect a bad night eventually.

FAQ

Is cordoning a node the same as draining it?

No. Cordoning only blocks new pods from being scheduled, and everything already running keeps running. Draining cordons the node and then evicts its pods so they restart on other nodes.

How do I uncordon a node in Kubernetes?

Run kubectl uncordon <node-name>. The node becomes schedulable again and loses the SchedulingDisabled status. Pods that were evicted do not move back automatically, so restart a rollout if you want them rebalanced.

Why is my kubectl drain stuck?

The usual cause is a PodDisruptionBudget that would be violated by evicting a pod, so drain keeps retrying. Other causes are pods with no controller (need --force), pods with emptyDir data (need --delete-emptydir-data), and slow-terminating pods. Run kubectl get pdb -A first.

Do I need to drain a node before rebooting it?

For planned maintenance, yes. Draining lets pods terminate gracefully and respects PodDisruptionBudgets, while an abrupt shutdown relies on the kubelet's graceful node shutdown behavior. Drain, reboot, then uncordon.

Will drain remove DaemonSet pods?

No. Drain skips them because the DaemonSet controller would recreate them at once. You still have to pass --ignore-daemonsets, or drain refuses to continue.