Delivery Semantics: What Each Guarantee Costs

The guarantee is the commit order

Delivery semantics sound like a configuration flag. They are not. They are decided by two lines of your consumer loop and the order you put them in.

Consider a consumer that reads a batch, does the work, and commits its offset. There are exactly two orderings.

Commit first, then process. The moment you commit, the broker believes you are done with those messages. If the process dies between the commit and the work, nobody will ever hand you those messages again. They are gone. This is at-most-once: every message is delivered zero or one times, never twice, and the failure mode is silent data loss.

Process first, then commit. The work is finished before the broker is told. If the process dies between the work and the commit, the restart resumes at the old committed offset and hands you those messages again. This is at-least-once: every message is delivered one or more times, and the failure mode is duplicates. It is Kafka's default posture and it is what almost every real pipeline runs.

Table

Same crash, two orderings, opposite damage. At-most-once loses silently; at-least-once duplicates loudly. Only one of those is recoverable.

ordering	timeline	crash lands here	damage	name
commit, then process	read -> COMMIT -> process -> ack	after the commit, before the work	messages the broker thinks are done but nothing happened to	at-most-once
process, then commit	read -> process -> COMMIT -> ack	after the work, before the commit	messages processed once, then handed over again on restart	at-least-once

Reading the damage out of the metadata

After a crash you have two artifacts: the consumer's own work log (which offsets it actually finished) and its commit log (what it told the broker). Line them up and the damage falls out of a comparison.

For an at-least-once consumer, look for offsets at or above the committed offset that were already processed successfully. Those are the redeliveries: the work happened, the broker was never told, so the restart will do the work again. Remember that the committed offset is the next one to read, so message_offset >= committed_offset is the correct boundary, not >.

For an at-most-once consumer, look for offsets below the committed offset that were never processed successfully. Those are the losses: the broker was told they were consumed and they never were. No restart brings them back, because from the cluster's point of view there is nothing to redeliver.

Exactly-once, honestly

Kafka does offer exactly-once semantics through transactions plus isolation.level=read_committed, and Flink and Kafka Streams build on it. That is senior awareness. What a junior candidate must be able to say, in one sentence, is the line the whole rest of this level is built on:

True exactly-once delivery is impossible. At-least-once delivery plus an idempotent sink is effectively exactly-once.

That is why Level 8 spent a whole module on making a load idempotent, and it is why the next module applies a changelog with an upsert rather than a blind insert. You do not prevent the duplicate. You make the duplicate harmless.

Two producer-side settings round out the answer. enable.idempotence (on by default since Kafka 3.0) makes the producer's own retries safe, so a network hiccup does not turn one send into two records. And acks trades latency against durability: acks=0 never waits, acks=1 waits for the leader only and loses data if that leader dies before replicating, acks=all waits for the in-sync replicas and is what you use for anything you would be embarrassed to lose.

Common mistake: treating at-least-once as "the safe one" and stopping there. At-least-once is only safe if the thing downstream can absorb a repeat. Sending a payment email twice, incrementing a counter twice, or appending to a ledger twice are all real incidents. The guarantee is half the answer and the sink is the other half.

Interview nuance: when you are asked "which delivery semantic would you choose", the answer that scores is not a semantic, it is a pair. Say at-least-once plus an idempotent sink keyed on something stable, and then name the key you would use. An interviewer is checking whether you know that duplicates are a design input rather than a bug to be argued away.

On a real platform this differs. Here you reconstruct the crash from two tables after the fact. On a real cluster you would not reconstruct anything, because the consumer's own code decides the ordering and you read the outcome from your sink. The tables here stand in for what a real consumer emits: a processing log or DLQ table on one side, and the broker's __consumer_offsets topic on the other. On AWS the same reasoning applies unchanged to a Kinesis consumer's checkpoint in its DynamoDB lease table, which is the direct analogue of a committed offset.