Partitioning, Ordering & Keys

One ordering guarantee, and it burns candidates

Kafka gives you exactly one ordering guarantee: records are totally ordered within a partition, and there is no ordering across partitions. A partition is a single append-only sequence, so offset order equals arrival order there. But a topic with 100 partitions is 100 independent sequences interleaved by wall-clock chance. If event X lands in partition 4 and event Y lands in partition 7, Kafka makes no promise about whether a consumer sees X or Y first. There is no global clock and no global sequence.

This is not a limitation to work around; it is the direct cost of parallelism. The only reason Kafka scales horizontally is that partitions are independent, so anything needing global order would need a single partition, capping you at one broker's throughput and one consumer.

The key is correctness

A producer computes the partition as hash(key) mod partition_count (default murmur2). So all records with the same key go to the same partition and are totally ordered relative to each other. Correctness reduces to one question: which events must be seen in order relative to each other? Whatever that set is, it must share a key.

• Bank account: key by account_id, so deposit-then-withdraw for one account is never reordered into overdraft.
• Order lifecycle: key by order_id, so created -> paid -> shipped stays monotonic.
• Chat: key by conversation_id, so a room's messages stay in order even though rooms interleave.

Interview nuance: the deadliest trap is assuming global ordering. Candidates say "Kafka keeps events ordered" and design a ledger that reads events across partitions expecting chronological order. It will silently apply a withdrawal before its deposit under load. The senior framing: order is per-partition only, so causally related events must be co-keyed.

A second trap: changing partition count breaks key-to-partition stability. Because the mapping is hash(key) mod N, changing N remaps most keys. New events for account_42 land in a different partition than the old ones, so its historical order splits across two partitions for the migration window. That is why partition count is effectively immutable in practice; you over-provision up front. (Partitions can be added, never removed, and even adding reshuffles the hash.)

The hot key

One account_id (a celebrity, an omnibus account, a viral post) can flood its partition while others idle. You cannot just split it, because splitting breaks the ordering you keyed for. Options, in order of preference:

Hot key "acct_42" floods partition 3:
 (a) Compound key: hash(account_id + sub_stream) -> spread across a few
     partitions, but ordering now only holds within each sub-stream.
 (b) Salting: key = account_id + (0..k) -> k partitions, then a downstream
     merge/serializer re-establishes per-account order by sequence number.
 (c) Accept it: if the hot key truly needs strict single-stream order,
     one partition is the ceiling; scale vertically and isolate it.

Every mitigation trades ordering scope for throughput. You either keep strict per-account order (one partition, capped throughput) or widen the key and downgrade to per-sub-stream order plus reassembly.

Recap: Kafka orders within a partition only, the partition is chosen by hash(key) mod N so causally related events must share a key; there is no global order; changing partition count remaps keys and breaks ordering, so partition count is fixed up front; and a hot key forces a choice between strict order (one partition, limited throughput) and salting/compound keys (more throughput, weaker ordering plus reassembly).