Latency Numbers Every Engineer Should Know
Memorize the latency ladder, unit conversions, object sizes, and single-machine ceilings, and turn each constant into a design decision.
The constants that make your math credible
Fast, credible estimation rests on a small set of memorized constants. If you quote a same-datacenter round trip as 50 ms or a memory read as 1 ms, every downstream number is wrong by orders of magnitude and the interviewer stops trusting your math. This lesson is the cheat-sheet.
The latency ladder
Rounded, order-of-magnitude, the numbers that matter:
L1 cache reference: ~1 ns. The baseline every other number on this ladder is measured against.
The key ratios to internalize: memory is roughly 1,000x faster than an SSD random read, an SSD is roughly 100x faster than an HDD seek, a same-datacenter round trip (~0.5 ms) is roughly 100x to 300x faster than a cross-region round trip. These ratios are why you cache in memory, why you avoid random disk seeks, and why you keep chatty request sequences within one region.
Interview nuance: the practical takeaway interviewers want is not the exact nanoseconds but the design consequence. "Cross-region is ~100 ms, so a synchronous read-your-writes across regions will feel slow; I will serve reads from a regional replica and replicate asynchronously" is the sentence that earns the point.
Units, time, and object sizes
1 KB ~= 10^3 bytes 1 MB ~= 10^6 1 GB ~= 10^9 1 TB ~= 10^12
1 day ~= 86,400 s ~= ~10^5 s
1 month ~= 2.5M s ~= 2.5 x 10^6 s
1 year ~= 31.5M s ~= ~3 x 10^7 s
a text message / tweet ~ 100s of bytes to ~1 KB
a database row (metadata) ~ 1 KB
a compressed web page ~ 100s of KB
a photo (post-compression) ~ 1-2 MB
a minute of video (SD/HD) ~ 5-15 MB
Single-machine ceilings
The rough capacities you assume before proving otherwise:
Tuned app server (stateless) ~ 10k-50k QPS
Redis / in-memory cache node ~ 100k+ ops/sec
Single relational DB primary ~ few k writes/sec, tens of k reads/sec
One server's live connections ~ 100k+ WebSockets (tuned)
These ceilings turn a QPS number into a server count in one step: 30k peak read QPS at ~10k QPS/server means 3 to 4 servers plus headroom; 1M write QPS against a ~5k-writes/sec DB node means you need ~200 shards, which immediately justifies a horizontally sharded datastore.
Interview nuance: you will not be graded on decimal precision. You will be graded on whether your quoted numbers are within an order of magnitude and whether you convert them into a decision (cache here, shard there, replicate this way). Being off by 1000x on a single constant can invalidate an otherwise good design.
Recap: memorize the latency ladder (memory ~100 ns, SSD ~100 us, same-DC ~0.5 ms, cross-region ~50 to 150 ms), the day/month-to-seconds conversions, typical object sizes, and single-machine ceilings, then always translate a number into a design decision.
From memory, place each operation on its rung of the latency ladder.
Apply
Your turn
The task this lesson builds to.
Order these by latency from memory and give rough magnitudes: L1/RAM read, SSD read, same-datacenter round trip, cross-region round trip, disk seek.
Think about
- What is the rough order of magnitude at each rung of the latency ladder?
- How do you convert one day and one month into seconds for QPS math?
- What single-machine ceilings (QPS, connections) do you assume?
Practice
Make it stick
A second problem on the same idea, so it survives past today.
Explain how you would use the latency ladder to justify a design decision in a global e-commerce checkout with users in the US, EU, and Asia, where inventory is the source of truth in a single US region. Give a concrete latency budget and the tradeoff you accept.
Think about
- How many cross-region round trips can a sub-second checkout afford, and what does each one cost from Asia?
- Which reads can move to regional replicas and caches, and which single write must stay authoritative?
- What consistency tradeoff do you accept on inventory, and what compensating action covers the edge case?