Kubernetes HPA: Horizontal Pod Autoscaling Explained with Real Examples

Your service handles 100 requests per second at 3am and 5,000 at 10am. If you run a fixed number of replicas, you're either wasting money or dropping requests. Kubernetes Horizontal Pod Autoscaling (HPA) solves this by adjusting the replica count automatically based on observed metrics. But set it up naively and you get thrashing, zombie replicas, and a metric feedback loop that makes things worse.

Here's how HPA actually works, how to configure it correctly, and the pitfalls that bite everyone in production.

1. How HPA Works Under the Hood

HPA is a control loop in the kube-controller-manager. Every --horizontal-pod-autoscaler-sync-period (default 15 seconds), it fetches metrics for your pods, computes the desired replica count, and writes it back to the Deployment or StatefulSet.

The core formula is simple:

desiredReplicas = ceil(currentReplicas × (currentMetricValue / desiredMetricValue))

If your target is 50% CPU and pods are at 75%, HPA scales up by a factor of 1.5. If they drop to 25%, it scales down by half. Two things make this formula tricky in practice:

Metrics lag. CPU metrics are averaged over the last 1–2 minutes, so HPA reacts to the past, not the present. Spiky traffic needs headroom or proactive scaling.
Scale-down is slow on purpose. By default HPA waits 5 minutes of sustained low utilization before shrinking. This prevents the classic "scale down, spike, scale up" thrash loop.

HPA doesn't know anything about your request rate, queue depth, or business metrics — unless you feed it to the metrics API. That's the key insight: CPU alone is usually the wrong metric.

2. Prerequisite: Install metrics-server

HPA needs the metrics.k8s.io API, which is provided by metrics-server. Many clusters — especially kubeadm and cloud-provider ones — don't ship it by default.

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

# Verify it works
kubectl top nodes
kubectl top pods -n default

If kubectl top errors with metrics not available yet, give it a minute. If it errors with connection refused on 10250, you're hitting the classic issue: metrics-server can't reach the kubelet because of TLS or network policy. For test clusters, the common fix is adding --kubelet-insecure-tls to the deployment args — never do that in production.

3. Your First HPA: CPU-Based Scaling

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 60

This scales the api Deployment between 2 and 10 replicas, keeping average CPU utilization around 60%. Note the API version: autoscaling/v2. The old v1 only supports CPU and has been deprecated — everything you see written with autoscaling/v2beta2 should now use autoscaling/v2.

Apply it and watch:

kubectl apply -f hpa.yaml
kubectl get hpa -w
# NAME    REFERENCE      TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
# api-hpa Deployment/api 45%/60%   2         10        2          3m

Now generate load and watch replicas climb:

kubectl run load-generator --image=busybox -- /bin/sh -c \
  "while true; do wget -q -O- http://api.default.svc:8080/ >/dev/null; done"

Within a minute you'll see TARGETS cross 60% and replicas increase. That's HPA working — but CPU scaling has a serious blind spot.

4. Why CPU-Only Scaling Fails

CPU utilization is a lagging indicator of actual user load. Three production scenarios where CPU-only HPA lets you down:

Request queuing. Your app accepts requests, puts them on a queue, and processes them in batches. CPU stays low while the queue grows — users wait, replicas don't.
I/O-bound workloads. Database queries, Redis calls, file I/O — these burn no CPU but add latency. CPU says "fine", p99 says "on fire".
Background jobs. A cron-like job inside your pods spikes CPU at 3am and HPA scales up for a workload that doesn't serve traffic. Money burned.

The fix: scale on custom or external metrics that actually reflect demand.

5. Custom Metrics: Scale on What Matters

For custom metrics you need a metrics adapter — the most common is prometheus-adapter, which exposes Prometheus queries through the custom.metrics.k8s.io API. Once it's installed, you can scale on request rate:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-http-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 2
  maxReplicas: 20
  metrics:
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: 500

This keeps each pod at roughly 500 req/s. The type: Pods metric aggregates a per-pod metric and divides by the pod count — perfect for request rates exposed via Prometheus client libraries.

The rule of thumb: scale on the metric that directly maps to capacity. For an API gateway, that's requests per second. For a queue consumer, that's queue depth (type: Object metric targeting the queue). For a WebSocket server, that's active connections. CPU is a proxy; these are the real thing.

6. Scaling Policies: Tame the Thrash

Default HPA behavior scales up immediately but waits 5 minutes before scaling down. Sometimes you want different behavior — fast scale-up for flash traffic, or conservative scale-down to avoid killing warm pods:

spec:
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
      - type: Percent
        value: 100
        periodSeconds: 15
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Pods
        value: 1
        periodSeconds: 60

This says: scale up by up to 100% every 15 seconds (double capacity in a minute), but scale down at most 1 pod per minute after a 5-minute stabilization window. Your app never falls over under a spike, and you don't shed capacity just because one poll caught a dip.

7. Production Pitfalls

These are the mistakes I see most often — and have made myself:

No resource requests. HPA's CPU utilization is computed against requests.cpu, not the limit. If you forget to set requests, HPA either refuses to compute or misbehaves. Always set requests on every container you autoscale.
Scaling based on a single pod's metrics. One hot pod skews the average. Use AverageValue with a sane threshold and check kubectl describe hpa for per-pod metric breakdowns when debugging.
maxReplicas too low. If your HPA hits the ceiling during a launch, you get dropped requests and no signal until it's too late. Set alerts on hpa_max_replicas being reached.
Autoscaling stateful workloads. HPA works on StatefulSets, but scaling a database or a stateful cache is how you corrupt data. Autoscale stateless tiers; scale stateful tiers manually or with operators.
Ignoring pod startup time. If your app takes 60 seconds to become ready and traffic spikes, HPA adds pods but they're not serving for a minute. Pair HPA with startupProbe and aggressive scale-up policies, or use KEDA's HTTP add-on for preemptive scaling.
Forgetting the cluster can't actually fit more pods. HPA will happily target 20 replicas while the node pool only fits 8. For cloud clusters, pair HPA with Cluster Autoscaler so nodes scale with pods — otherwise you're autoscaling into a wall.

8. Debugging HPA

# See the exact replica calculations and events
kubectl describe hpa api-hpa

# Check the raw metrics HPA sees
kubectl get --raw /apis/metrics.k8s.io/v1beta1/namespaces/default/pods

# Check custom metrics availability
kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 | head

# Watch status conditions
kubectl get hpa api-hpa -o jsonpath='{.status.conditions}'

If kubectl describe hpa shows failed to get cpu utilization: unable to get metrics, it's almost always metrics-server. If it shows invalid metrics, your custom metric query returns nothing — check the prometheus-adapter config, not the HPA.

Summary

HPA adjusts replica count from the metrics API using a simple ratio formula — but metrics lag, so CPU-only autoscaling under-reacts to real load.
Install metrics-server first; use autoscaling/v2, never the old v1 API.
Scale on the metric that directly maps to capacity: req/s, queue depth, connections — not CPU as a proxy.
Configure behavior policies so scale-up is fast and scale-down is conservative.
Always set resource requests, set an alert on max replicas, and pair HPA with Cluster Autoscaler in the cloud.

Autoscaling done right means your infra bill tracks your traffic curve and your p99 stays flat during spikes. Start with CPU to learn the loop, then move to real demand metrics — that's the difference between autoscaling that saves money and autoscaling that saves your weekend.