autoscaling

Autoscaling

Autoscaling in Kubernetes is a control loop stacked on a control loop: the HPA watches a metric and adjusts replica count, the scheduler places those replicas, and the cluster autoscaler watches for unschedulable pods and adds nodes. Every layer depends on the one below reporting truthfully — an HPA scaling on a metric that doesn't reflect real load, or replicas whose requests don't reflect real usage, produces scaling decisions that look active but don't fix anything.

Autoscaling amplifies whatever signal you point it at — a good signal gives you elastic capacity, a bad one gives you expensive noise. Get requests right first; every autoscaler downstream is only as accurate as the numbers it's reading.

1. Scale on the metric that actually predicts saturation

CPU utilization is the default HPA metric because it's always available, not because it's usually the right one. A queue-processing service saturates on queue depth; an API saturates on request latency or in-flight requests; a memory-bound service doesn't reflect load in CPU at all. Scaling on the wrong metric means the HPA reacts late or not at all to the thing that's actually hurting users.

• CPU/memory (autoscaling/v2 resource metrics) work when the workload's bottleneck genuinely is CPU or memory — verify this, don't assume it.
• Custom and external metrics (via a metrics adapter — Prometheus Adapter, KEDA, cloud-provider metrics) let you scale on queue depth, request rate, or a business metric — worth the setup cost when the resource metrics don't correlate with actual saturation.
• Target utilization should leave headroom for the scale-up lag — if pods take 30s to become Ready, targeting 90% CPU means you're already in trouble before new replicas can help.

Done when: the HPA's target metric has been checked against real saturation data, not chosen by default.

2. Don't run HPA and VPA on the same metric for the same workload

HPA changes replica count; VPA changes each replica's requests/limits. Pointed at the same resource dimension on the same workload, they fight — VPA resizing a pod's CPU request changes the value HPA's percentage target is computed against, producing scaling decisions neither system intended.