Picture a hotel with a "No Vacancy" sign lit up out front. Every room is reserved. Walk the hallways, though, and almost every door is open, the beds are still made, and the towels are folded. Guests booked rooms for the whole family just in case, then one person showed up. Meanwhile, the manager is out back pouring the foundation for a new wing, because according to the reservation book, the hotel is full.
That hotel is the average Kubernetes cluster. (I used AI tools to help research and draft this post, and checked the Kubernetes behavior against the official documentation and the numbers against the report they come from.)
The number
Cast AI publishes a yearly report on how Kubernetes clusters use what they pay for. Its 2026 State of Kubernetes Optimization Report, released in April and drawn from tens of thousands of clusters, found that average CPU utilization across clusters was 8% in 2025. Memory utilization was 20%. GPUs, the most expensive hardware in the building, averaged 5%.
The trend is going the wrong way. The same company's earlier reports put average CPU utilization at 13% in the 2024 edition and 10% in the 2025 edition.

Average utilization reported by Cast AI. Each bar is the share of paid-for capacity actually doing work.
One fair caveat: Cast AI sells Kubernetes cost optimization, and its press releases describe the data as coming from clusters before optimization ("non-optimized Kubernetes clusters" in 2026). A team that has already tuned its cluster will look better. But "tens of thousands of clusters" is a big sample of what clusters look like when nobody has gone back to tune them, which is most of them.
So if your cluster looks like the average one, about 92% of the CPU you're paying for is doing nothing at any given moment.
The front desk counts reservations
Here's the mechanism, and it's working exactly as designed.
When you deploy a container to Kubernetes, you can declare a resource request: "this container needs half a CPU and 512 MB of memory." That's the reservation. The Kubernetes scheduler is the front desk. It places each pod on a node only if the node has enough unreserved capacity to cover the pod's requests.
The Kubernetes documentation is direct about what the front desk ignores: even when actual memory or CPU usage on a node is very low, the scheduler still refuses to place a pod there if the requested amounts don't fit. It counts reservations. It never looks in the rooms.
That's deliberate. The docs explain that it protects a node from running short when usage climbs later, like during a daily traffic peak. A hotel that gave away reserved rooms to walk-ins would be in trouble the moment the guests arrived.
The problem is the next step. When a new pod can't fit anywhere, the cluster autoscaler adds a node. Both of the autoscalers the Kubernetes project sponsors provision new nodes for pods that can't be scheduled. So padded reservations don't just waste the rooms you have. They make the manager build new wings.

What the scheduler sees versus what the CPU does. The gap is what you pay for and don't use. (Illustrative numbers)
Why everyone over-books
Nobody pads requests out of carelessness. Every incentive points the same way.
Running out hurts more than over-booking. If a container uses more memory than its limit allows, the kernel kills it (an out-of-memory kill). If it hits its CPU limit, the kernel throttles it and your latency climbs. Being short shows up as an incident. Being long shows up as a bigger bill that nobody gets paged for.
Nobody knows the right number on day one. Requests get set when a service launches, often copied from another service or from a tutorial, and rarely revisited. Traffic changes, code changes, and the reservation stays where it was.
Peaks get sized, averages get paid. A service that needs two CPUs for ten minutes a day often requests two CPUs all day.
Some of the empty space is on purpose. You run three replicas across zones so a zone failure doesn't take you down. You keep headroom for traffic spikes and for the moment a node dies and its pods need somewhere to land. That headroom is insurance you chose. The goal is to keep the insurance and lose the accidental part.
How to give the rooms back
None of this needs new tools. It needs the reservation book to match reality.
Measure requests against usage, per workload. For each deployment, compare what it requests with what it actually uses over a couple of weeks, including the peaks. The gap is where your money goes. Most teams find a handful of services account for most of it.
Let the Vertical Pod Autoscaler recommend first. The Vertical Pod Autoscaler (VPA) watches actual usage and recommends requests. Run it in "Off" mode to get recommendations without letting it change anything, then apply them deliberately, starting with the biggest gaps.
Treat CPU and memory differently. Running short on memory gets a container killed, so keep memory requests close to real peak usage. Running short on CPU slows a container down, so CPU requests can sit closer to typical usage, especially for services that don't sit on a latency-sensitive path.
Let the nodes follow the requests down. Shrinking requests only saves money if the cluster removes the nodes that empty out. Both sponsored autoscalers consolidate nodes that are no longer well used, so make sure consolidation is turned on and isn't blocked by pods that can't be moved.
Scale replicas with demand. A Horizontal Pod Autoscaler adds and removes replicas as traffic changes, so you reserve capacity for the traffic you have instead of the peak you might have.
Write down the headroom you're keeping on purpose. Decide how many spare replicas, zones, and nodes your reliability needs, and keep them. Everything above that line is fair game.
Look hardest at GPUs. At 5% average utilization, a GPU node is the most expensive empty room in the building. Share GPUs between workloads where your hardware supports it, and scale development and batch GPU pools to zero when nobody is using them.
The "No Vacancy" sign is lying
A full reservation book feels like a busy hotel, but the two can be far apart. Your cluster's scheduler is doing its job: it keeps promises. The trouble is that most of those promises were made once, by someone guessing, and never checked again.
Full disclosure: I work at Control Plane, which runs containers across clouds so teams don't have to manage cluster capacity themselves. You don't need any of that to act on this post. Pick your three most expensive deployments, put their requests next to their real usage, and see how many beds are still made.
Originally published on Medium.