Generating a personalized feed for 500 million daily active users is one of the hardest distributed systems problems. The core challenge: when user A posts, how does it appear in the feeds of all A's followers — potentially millions of them — fast?

Two Fundamental Approaches

Fanout-on-write (push model): when a user posts, immediately write that post into every follower's feed cache. Feed reads are instant — just read pre-computed feed. Cost: one write amplifies into millions of writes for celebrity accounts.

Fanout-on-read (pull model): store posts in a user's post table. On feed load, fetch posts from all accounts you follow and merge/sort them. No write amplification, but feed generation is slow and expensive — you're doing a fan-in merge at read time.

The Hybrid Approach (What Twitter Actually Does)

Use fanout-on-write for normal users (< 10K followers) — fast feed reads, manageable write load. For celebrity accounts (millions of followers), skip the fanout entirely. Instead, on feed load, fetch the celebrity's recent posts separately and merge them into the pre-computed feed. This caps the write amplification spike without sacrificing read latency for normal users.

async function getFeed(userId: string): Promise<Post[]> {
  // 1. Pre-computed feed from Redis (fanout-on-write for normal accounts)
  const cachedFeed = await redis.lrange(`feed:${userId}`, 0, 200)

  // 2. Merge in celebrity posts (fanout-on-read for high-follower accounts)
  const celebrities = await getCelebrityFollows(userId)
  const celebPosts = await Promise.all(
    celebrities.map(c => getRecentPosts(c.id, limit: 20))
  )

  return rankAndMerge([...cachedFeed, ...celebPosts.flat()])
}

Feed Ranking

Raw chronological feeds were abandoned years ago. Modern feeds rank by engagement signals: recency, likes, comments, shares, time-spent, relationship strength (how often you interact with this person), content type preference. This ranking runs as a lightweight ML model inference on the merged candidate set before serving.

Data Stores and Infrastructure

• ▸Post storage: Cassandra or DynamoDB — high write throughput, time-series access pattern
• ▸Feed cache: Redis sorted sets (score = timestamp or rank score) per user
• ▸Social graph: Neo4j or a purpose-built graph DB for follower/following relationships
• ▸Fanout workers: Kafka consumers that process post events and write to follower feed caches
• ▸Media: separate CDN pipeline (images/video are not stored with post metadata)
• ▸Feed size cap: keep only last 800 posts in feed cache — older content fetched on-demand