Skip to main content

Design a Distributed Unique ID Generator (Snowflake)

Level 10: Level 10: Applied Case Studiesmedium30 minsnowflakeid-generationclocks

Budget the 64 bits (timestamp high for sortability, worker id, sequence), which lets every node mint millions of unique IDs per second with zero coordination; defend the clock by refusing to issue on a backward jump; and remember that sortability trades away unpredictability, so hide raw ids when enumeration is a threat.

Coordination-free is the whole point

The job is to hand out 64-bit, globally unique, roughly time-sortable IDs at millions per second without any node talking to any other node on the request path. Coordination-free is the whole point: a central sequence server would be a bottleneck and a single point of failure.

The Snowflake bit budget

Snowflake's trick is to partition the ID space by bit budget so each node can mint IDs alone. A common 64-bit layout: 1 sign bit (unused, kept 0 so the number is positive), 41 bits of millisecond timestamp (since a custom epoch), 10 bits of machine/worker id, and 12 bits of a per-millisecond sequence counter. Do the arithmetic, because interviewers ask. 41 bits of ms is 2^41 milliseconds which is about 69 years of range from your epoch. 10 bits of worker id is 1,024 nodes. 12 bits of sequence is 4,096 ids per node per millisecond, which is about 4.096M ids per node per second, times 1,024 nodes is over 4 billion/sec of theoretical ceiling. You can rebudget the bits (fewer worker bits, more sequence) to match your fleet.

The ID is time-sortable because the timestamp occupies the high bits: sort the 64-bit integers and you get roughly chronological order, which is why these make excellent clustered primary keys. Within a single millisecond the sequence counter breaks ties and guarantees uniqueness; if the sequence overflows (more than 4,096 ids in one ms on one node), the generator waits (busy-spins) until the next millisecond.

Interview nuance: compare the alternatives out loud. UUIDv4 is random 128-bit: trivially coordination-free and unpredictable, but not sortable, and as a clustered index key its randomness scatters writes across the B-tree and fragments the index (the classic wrong turn). UUIDv7 and ULID fix that by putting a timestamp in the high bits (like Snowflake, but 128-bit and needing no worker-id assignment). DB auto-increment is perfectly sortable and compact but needs central coordination and does not shard. A ticket server (a dedicated ID service) centralizes allocation and reintroduces the bottleneck Snowflake exists to avoid. Snowflake wins when you want compact, sortable, coordination-free 64-bit keys and can manage worker ids.

The clock is the weakness

Because the timestamp is in the high bits, if a node's clock jumps backward (NTP correction, VM migration), it could generate an ID with a smaller timestamp than one it already issued, risking a duplicate or breaking monotonicity. The standard defense: track the last-issued timestamp; if the current clock is behind it, refuse to issue IDs (throw, or wait) until the clock catches up, rather than emit a possibly-duplicate ID. Depend on NTP to keep clocks disciplined, but never trust it blindly.

Worker-id assignment is the other operational detail. Each node needs a unique 10-bit id. Assign it via a coordination service (ZooKeeper or etcd) that leases ids, or from a config/orchestrator on startup. Exhaustion (more than 1,024 live nodes) means you rebudget bits or recycle ids from dead nodes.

Table
64 bits total, and the field ORDER is the design: timestamp sits highest so a plain integer sort is roughly a time sort. Move it lower and you keep uniqueness but lose sortability, which is the whole reason to prefer this over a random UUID.
FieldBitsWhat it buysRuns out at
Sign1Always 0, so the id stays a positive signed 64-bit integerReserved, never used
Timestamp (ms since a custom epoch)41Rough sortability by creation time, because it occupies the high bits≈ 69 years after the epoch you pick
Worker id10Uniqueness across nodes with no coordination per id1,024 simultaneously live nodes
Sequence12Uniqueness within one millisecond on one node4,096 ids per node per millisecond
64 bits total, and the field ORDER is the design: timestamp sits highest so a plain integer sort is roughly a time sort. Move it lower and you keep uniqueness but lose sortability, which is the whole reason to prefer this over a random UUID.

There is a real tension: sortability leaks information. A time-sortable ID reveals creation time and, worse, sequential-ish IDs let an attacker enumerate or estimate volume ("how many orders did they get today"). If unpredictability matters (public-facing resource ids), do not expose the raw sortable id; use a random UUID externally and keep the Snowflake id internal, or add a non-sequential public slug.

Recap: budget the 64 bits (timestamp high for sortability, worker id, sequence), which lets every node mint millions of unique IDs per second with zero coordination; defend the clock by refusing to issue on a backward jump; and remember that sortability trades away unpredictability, so hide raw ids when enumeration is a threat.

Apply

Your turn

The task this lesson builds to.

Design a service issuing 64-bit, time-sortable, globally unique IDs at millions/sec without central coordination.

Think about

  1. How do you budget the bits (timestamp, worker, sequence)?
  2. How do you handle clock skew and rollback?
  3. What is the sortability vs unpredictability tension?

Practice

Make it stick

A second problem on the same idea, so it survives past today.

Design the primary-key generation strategy for Discord's message store, which writes billions of messages/day into Cassandra, needs ids that sort by creation time so a channel's history can be range-scanned efficiently, and must let clients derive a message's approximate timestamp from its id offline, all without a central sequence service.

Think about

  1. How does a time-sortable id become the Cassandra clustering key and cursor?
  2. How do clients decode the timestamp offline from the id?
  3. Why is per-channel ordering enough despite small cross-worker clock differences?