Autoscaling: Reactive, Event-Driven & Predictive

Match capacity to demand, and respect the lag

Autoscaling is the machinery that grows and shrinks a fleet so you pay for roughly what you use while still meeting your SLOs. There are three layers, and interviewers want you to name them distinctly.

Horizontal Pod/instance autoscaling (HPA) adds or removes replicas based on a metric. The default metric is CPU or memory utilization: target 60% CPU, add pods when the average climbs above that. The problem is that CPU is a lagging signal: by the time CPU is pegged, requests are already queuing and your p99 is already blown. Better is to scale on a leading business metric: requests-per-second per pod, in-flight concurrency, or, best of all for async workers, queue depth / consumer lag. If a Kafka or SQS backlog is growing, you need workers now, before any CPU number moves.

Event-driven autoscaling is exactly this idea productized. KEDA scales a deployment directly off external event sources: Kafka lag, SQS queue length, Redis list size, Prometheus queries. A worker fleet can even scale to zero when the queue is empty. This reacts to the cause (work arriving) rather than the symptom (CPU rising), buying precious lead time.

Below the pod layer sits the cluster/node autoscaler. HPA asking for 40 more pods does nothing if there is no node to place them on; the Cluster Autoscaler (or Karpenter) provisions new VMs when pods are unschedulable. Separately, the Vertical Pod Autoscaler (VPA) right-sizes each pod's CPU/memory requests. HPA and VPA on the same metric fight each other, so keep them on different signals.

Scaling lag: the concept that separates a senior answer

Reactive autoscaling has an unavoidable pipeline of delays: metric scrape interval (15 to 60s) + controller decision/stabilization window + node provisioning (30 to 120s for a fresh VM) + container pull + app boot + JIT/cache warmup + health-check pass. That is often 2 to 5 minutes end to end. A traffic burst that arrives in 20 seconds will overwhelm you long before new capacity is ready. Reactive scaling always trails a fast burst.

Two tools hide the lag. Warm pools keep pre-booted, pre-warmed instances parked so a scale-out is just "attach," collapsing minutes to seconds. Scheduled / predictive pre-scaling grows the fleet ahead of a known pattern: if traffic 10x's every day at 9am, a scheduled scaler raises the floor at 8:50. Predictive autoscalers learn the daily/weekly curve and pre-provision automatically.

Interview nuance: the trap is claiming "autoscaling handles spikes" full stop. The correct framing: autoscaling handles sustained load changes and gradual ramps well, but for sharp bursts you must either pre-scale (if predictable) or hold headroom (run at 60% not 95%) so the existing fleet absorbs the burst while new capacity boots.

  burst arrives ->  |####| traffic
  reactive:         scrape(30s)+decide(30s)+boot(90s)+warm(30s) = ~3min late
  warm pool:        attach pre-booted node = ~15s
  scheduled:        capacity already up at 8:50 for the 9am spike

Recap: scale on leading signals (queue depth via KEDA, RPS) not just lagging CPU, layer HPA + cluster autoscaler + VPA, and because reactive scaling always trails a fast burst by 2 to 5 minutes of lag, hide that lag with warm pools, scheduled pre-scaling, and standing headroom.