Headroom — CPU limits that resize to share unrequested capacity

Why it exists

Static limits force a bad choice.
Headroom removes the choice.

No mystery throttling

Tight limits throttle pods on idle nodes — the classic unexplainable latency. Headroom raises limits exactly when the node has unused capacity to give.

Isolation when it matters

As requests fill the node, every limit converges to its request — the same fair share cpu.weight enforces under contention. Coherent at both extremes.

Safe by construction

Controller dies? Limits freeze at their last values and the kernel still shares fairly. No failure mode is worse than not running Headroom at all.

How it works

Requests-driven. Deterministic. Debuggable.

Your CPU request is already a real quality-of-service guarantee: the scheduler books it, and the kernel enforces it as a floor under contention (cpu.weight). The only thing that ever throttles is the quota — the limit. So the only question is: what should the ceiling be right now?

slack(node) = allocatable_cpu − Σ requests(all pods on node) limit(pod) = request × (1 + slack / Σ requests(managed pods))

Watch scheduling events

Pods bind, resize, or leave a node — that changes booked capacity. Nothing else does. No metrics pipeline, no usage sampling.

Recompute the node

One pass distributes the node's unused CPU across managed pods — each gets a fair share, proportional to its request — with a floor, caps, and hysteresis.

Resize in place

New limits apply through the GA in-place pod resize subresource: a live cgroup write. No restarts, and kubectl get pod shows the truth.

Every limit is explainable from observable inputs — an annotation, an event, and metrics on every change. Full rationale in the design doc.

The honest part

Should you even run this?

RUN IT WHEN

Hostile or contractual multi-tenancy — tenants who must not degrade each other, or SLAs you have to bound.
Blast-radius bounds — a hard ceiling caps what a runaway spin loop can take from its neighbors.
Self-service QoS tiers — a tenant's CPU request becomes their service tier: guaranteed floor, predictable ceiling. Continuously sized, not a menu of fixed shapes.
Tenants fresh off VMs — workloads that size thread pools from visible CPUs, and nproc reports the whole node. Point the Downward API at limits.cpu; Headroom makes that number real before bin-packing exposes the lie.
CI & batch on shared nodes — compile bursts on idle nodes are the showcase case.

SKIP IT WHEN

Trusted, single-tenant clusters — just omit CPU limits (or set --cpu-cfs-quota=false). The kernel already shares fairly, and nothing ever throttles. You don't need Headroom.
Guaranteed or BestEffort pods — structurally excluded; resize can't change QoS class.
Gang-scheduled synchronous training — different ceilings per worker make stragglers. Opt out.
Static CPU Manager / Windows nodes — in-place resize is unavailable there.

Headroom is opt-in per namespace and earns its complexity only where CPU ceilings are a requirement, not a preference. The full matrix — workload classes, schedulers, VPA/HPA coexistence — is in the applicability guide. Someone enrolled your namespace? App teams start here.

ON AIR · v0.1.0

Headroom v0.1.0 is out — the first public release, Apache-2.0. The policy core is covered by table + property tests for every design invariant, and the controller installs via two Helm charts (release notes).

It defaults to dryRun: true — computing targets, annotating pods, and emitting metrics without issuing a single resize until you flip the switch. Start with the runbook preflight, follow priorities in the backlog, or star the repo.

Static limits force a bad choice.Headroom removes the choice.