Skip to main content

Design a News Feed / Timeline (Twitter)

Level 10: Level 10: Applied Case Studieshard40 minnews-feedfan-outcelebrity

Use a hybrid fan-out, push posts to normal followers' timelines and pull celebrities at read time, store post ids not bodies so deletes stay cheap, paginate by cursor, and accept async fan-out lag (seconds) as the freshness tradeoff.

The whole interview hinges on "fan-out"

A home timeline shows a user the recent posts of everyone they follow, newest first, in under 200ms. The entire problem is a read-vs-write cost tradeoff, and the whole interview hinges on the word "fan-out."

Fan-out-on-write (push)

When Alice posts, you immediately write that post id into the timeline cache of every follower. Reads become trivial: a follower's timeline is a precomputed list you slice with a cursor. The cost moves to write time. If Alice has 200 followers, one post is 200 small writes. That is fine until Alice is a celebrity with 50M followers, at which point a single tweet is 50M writes, a multi-minute fan-out that hammers the cache and delays delivery.

Fan-out-on-read (pull)

Store each post once keyed by author. When Bob loads his timeline, fetch the recent posts of everyone Bob follows and merge them at read time. Writes are cheap (one insert). Reads are expensive: if Bob follows 2,000 accounts you issue a scatter-gather across 2,000 authors and merge-sort on every timeline load. That blows the 200ms budget for active users.

The hybrid (the senior answer)

Fan-out-on-write for the common case, fan-out-on-read for celebrities. When you post, you push to normal followers' timelines. Accounts above a follower threshold (say 100K) are marked "celebrity" and are NOT pushed. At read time you take a user's precomputed timeline and merge in the recent posts of the handful of celebrities they follow, pulled live and cached briefly. Most users follow only a few celebrities, so the read-time merge is small and bounded. This caps write amplification and keeps reads fast.

Alice posts
  |
  +-- Alice is normal?  push post_id -> timeline:<each follower>   (fan-out-on-write)
  +-- Alice is celeb?   do nothing on write; readers pull her recent posts

Bob loads timeline:
  precomputed timeline:Bob   (Redis list of post_ids)
  + merge recent posts of celebs Bob follows (pulled + cached)
  -> rank -> hydrate post bodies -> return page

Storage, ranking, deletes

Posts live once in a partitioned store (Cassandra or a sharded SQL, partitioned by post id or author). Per-user timelines are Redis lists or sorted sets of post ids (not full bodies), capped to a few hundred entries. You hydrate bodies in a second batched lookup. Pagination uses an opaque cursor (last post id or a score), never OFFSET, which degrades linearly.

Chronological is a sorted set scored by timestamp. ML-ranked timelines change the shape: fan-out now delivers candidates, and a ranking service scores them per request using features (author affinity, recency, engagement). You keep fan-out as candidate generation and add a scoring layer.

Because post bodies are stored once and timelines hold only ids, a delete is a tombstone on the post; readers filter tombstoned ids at hydration. You do not chase 50M cached copies. This is exactly why timelines store ids, not bodies: it keeps the source of truth single and makes deletes and edits O(1).

Interview nuance: the consistency-vs-freshness tradeoff. Fan-out-on-write means a follower may see a post seconds after it is created (async fan-out lag). That is acceptable for a feed. Do not promise read-after-write on someone else's timeline.

Recap: use a hybrid, push posts to normal followers' timelines and pull celebrities at read time, store post ids not bodies so deletes stay cheap, and paginate by cursor.

Apply

Your turn

The task this lesson builds to.

Design a home timeline that shows a user the recent posts of everyone they follow, at read latency under 200ms.

Think about

  1. When do you fan out on write vs on read?
  2. How does a hybrid handle celebrity accounts?
  3. How do ranking and deletes/edits change the design?

Practice

Make it stick

A second problem on the same idea, so it survives past today.

Design the timeline for X (Twitter) during a live event like the World Cup final, where a single account (the official league account) with 90M followers posts a goal, and 30M users are refreshing their timeline in the same 60-second window. Explain how you keep both the write and read paths from collapsing.

Think about

  1. Why is this a read hot-key problem, not a write-throughput problem?
  2. How do hot-key replication, coalescing, and short local caches survive 500K read QPS on one key?
  3. How does ranking degrade gracefully under the spike?