Transactional Messaging: Outbox, Inbox & CDC

The dual-write problem

Sagas and event-driven systems depend on a step that looks trivial and is not: "update my database and publish an event." Doing both is the dual-write problem, and it has no atomic solution across two independent systems without a distributed transaction.

Consider the naive code: write the order row to Postgres, then publish OrderCreated to Kafka. Two failure orderings break you. If the service crashes after the DB commit but before the Kafka publish, the order exists but no event was ever sent: downstream systems never hear about it, a lost event. Flip the order (publish first, then write the DB) and a failed DB write leaves an event for an order that does not exist: a phantom event. You cannot wrap a Postgres commit and a Kafka publish in one atomic transaction, because they are separate systems with separate logs.

The transactional outbox

Make the event part of the same local database transaction as the business data:

BEGIN;
  INSERT INTO orders (...);                          -- business write
  INSERT INTO outbox (event_type, payload, ...);     -- event, SAME txn
COMMIT;                                              -- both or neither

The order row and the "there is an event to publish" fact commit atomically, in one local transaction in one database. A separate relay process reads unpublished outbox rows and publishes them to Kafka, marking them sent after the broker acknowledges.

Two ways to run the relay:

• Polling: periodically SELECT ... FROM outbox WHERE published = false and publish. Simple, works anywhere, but adds polling latency and query load, and needs FOR UPDATE SKIP LOCKED to avoid double-scanning under concurrency.
• Change Data Capture (CDC): Debezium tails the database's write-ahead log and streams committed changes to Kafka directly. No polling, low latency, low DB load, more infrastructure. The production default at scale.

At-least-once plus the inbox

The relay guarantees the event is published at least once: if it crashes after publishing but before marking the row sent, it republishes on restart. So consumers can receive duplicates. The inbox pattern closes this: the consumer records each processed event id in an inbox/dedup table inside the same transaction as its side effect, and skips any id it has already seen. At-least-once delivery plus an idempotent (inbox-backed) consumer equals effectively-once end-to-end processing: the strongest realistic guarantee.

Interview nuance: be precise that the outbox does not give exactly-once delivery. It converts "atomically write DB and publish" (impossible) into "atomically write DB and record intent to publish" (a single local transaction), then relies on at-least-once relay plus consumer idempotency. Interviewers love to hear the ordering-of-failures argument for why the naive dual write is broken.

Recap: writing the DB then publishing to Kafka is not atomic and either loses or fabricates events, so write the event into an outbox table in the same local transaction and let a relay (polling or Debezium CDC) publish it at least once, with a consumer-side inbox/dedup table making the end-to-end result effectively-once.