Definition
- Fanout-on-write (push) – On post, push a denormalized entry into each follower's precomputed timeline; read is a simple list lookup.
- Fanout-on-read (pull) – Store posts once; on read, pull and merge the followees' recent posts; writes are trivial.
- Write amplification – One post triggers N writes where N is the follower count; catastrophic for celebrities.
- Hybrid – Fanout-on-write for normal users (modest followers), fanout-on-read for celebrities (millions of followers).
- Celebrity threshold – The follower count above which a user is treated as a celebrity; trades write cost vs read cost.
Think of National Bank of India sending account-activity notices. Fanout-on-write stuffs a copy into every interested customer's mailbox immediately, so opening the mailbox is instant, but one event for a million customers means stuffing a million mailboxes. Fanout-on-read keeps one noticeboard and gathers relevant notices on demand, which is cheap to post but slow to read. The hybrid: stuff mailboxes for ordinary notices, and for notices that interest millions, fetch on demand and blend in.
Understanding the Concepts
What is the fundamental tension?
A feed read is extremely frequent and must be fast. Posting is rare but can affect millions of followers. You are trading where you pay the cost.
package com.bank.feed;
public class FeedTension {
public static void main(String[] args) {
System.out.println("Feed design tension:");
System.out.println(" Reads: millions per second, must be < 50ms");
System.out.println(" Writes: thousands per second, can tolerate seconds");
System.out.println();
System.out.println(" One post by user with 500 followers: 500 writes (manageable)");
System.out.println(" One post by celebrity with 5M followers: 5,000,000 writes (!)");
System.out.println();
System.out.println(" Trade-off: pay at write time or at read time?");
}
}
Expected output:
Feed design tension:
Reads: millions per second, must be < 50ms
Writes: thousands per second, can tolerate seconds
One post by celebrity with 5M followers: 5,000,000 writes!
How does fanout-on-write (push) work?
At post time, push a denormalized entry into each follower's precomputed timeline. Reading is a simple list lookup.
package com.bank.feed;
import java.util.*;
public class FanoutOnWrite {
private final Map<String, List<String>> timelines = new HashMap<>();
public void post(String author, String content, List<String> followers) {
String entry = author + ": " + content;
for (String follower : followers) {
timelines.computeIfAbsent(follower, k -> new ArrayList<>()).add(0, entry);
}
System.out.println("Posted by " + author + " -> pushed to " + followers.size() + " timelines");
}
public List<String> readFeed(String userId) {
return timelines.getOrDefault(userId, List.of());
}
public static void main(String[] args) {
var feed = new FanoutOnWrite();
feed.post("analyst-pankaj", "NBI Q4 results: strong growth",
List.of("rajesh", "priya", "vikram"));
System.out.println("Rajesh's feed: " + feed.readFeed("rajesh"));
}
}
Expected output:
Posted by analyst-pankaj -> pushed to 3 timelines
Rajesh's feed: [analyst-pankaj: NBI Q4 results: strong growth]
Pro: read is O(1) list lookup
Con: write amplification = follower count
How does fanout-on-read (pull) work?
Store posts once. On read, pull and merge the followees' recent posts.
package com.bank.feed;
import java.util.*;
public class FanoutOnRead {
private final Map<String, List<String>> posts = new HashMap<>();
private final Map<String, List<String>> follows = new HashMap<>();
public void post(String author, String content) {
posts.computeIfAbsent(author, k -> new ArrayList<>()).add(0, content);
System.out.println("Stored post by " + author + " (1 write, no fanout)");
}
public List<String> readFeed(String userId) {
List<String> feed = new ArrayList<>();
for (String followee : follows.getOrDefault(userId, List.of())) {
List<String> followeePosts = posts.getOrDefault(followee, List.of());
if (!followeePosts.isEmpty()) {
feed.add(followee + ": " + followeePosts.get(0));
}
}
return feed;
}
public static void main(String[] args) {
var feed = new FanoutOnRead();
feed.follows.put("rajesh", List.of("analyst-pankaj", "market-index"));
feed.post("analyst-pankaj", "NBI Q4 results: strong growth");
feed.post("market-index", "Sensex up 2% today");
System.out.println("Rajesh's feed: " + feed.readFeed("rajesh"));
}
}
Expected output:
Stored post by analyst-pankaj (1 write, no fanout)
Stored post by market-index (1 write, no fanout)
Rajesh's feed: [analyst-pankaj: NBI Q4 results, market-index: Sensex up 2%]
Pro: write is O(1)
Con: read is O(followees) fan-in merge -> slow at scale
Why does neither pure strategy work?
Pure push collapses on celebrities. Pure pull collapses on normal reads.
package com.bank.feed;
public class WhyNeitherWorks {
public static void main(String[] args) {
System.out.println("Pure fanout-on-write:");
System.out.println(" Normal user (500 followers): 500 writes -> fine");
System.out.println(" Celebrity (5M followers): 5,000,000 writes -> CATASTROPHIC");
System.out.println(" Write storm, slow, expensive, wastes effort on inactive followers");
System.out.println();
System.out.println("Pure fanout-on-read:");
System.out.println(" Every feed load: pull + merge from all followees -> SLOW");
System.out.println(" Rajesh follows 200 people -> merge 200 post lists on every read");
System.out.println(" Millions of reads -> massive fan-in load");
System.out.println();
System.out.println("Solution: HYBRID -> push for normal, pull for celebrities");
}
}
Expected output:
Pure push: celebrity = 5M writes per post -> catastrophic
Pure pull: every read = merge 200 lists -> slow
Solution: HYBRID
How does the hybrid work?
Normal users: push into followers' timelines. Celebrities: pull at read time and merge with precomputed timeline.
package com.bank.feed;
public class HybridFeed {
private static final int CELEBRITY_THRESHOLD = 10000;
public static void main(String[] args) {
int analystFollowers = 500;
int indexFollowers = 5_000_000;
System.out.println("Hybrid feed:");
System.out.println(" analyst-pankaj (" + analystFollowers + " followers): PUSH");
System.out.println(" -> Push into 500 timelines. Read: O(1) list lookup.");
System.out.println();
System.out.println(" market-index (" + indexFollowers + " followers): PULL");
System.out.println(" -> Store once. On read: merge cached celebrity posts.");
System.out.println();
System.out.println(" Rajesh's feed read:");
System.out.println(" 1. Read precomputed timeline (analyst posts already there)");
System.out.println(" 2. Pull cached market-index recent posts");
System.out.println(" 3. Merge and sort by time");
System.out.println(" = Fast reads + no celebrity write storm");
}
}
Expected output:
Hybrid feed:
analyst-pankaj (500 followers): PUSH into 500 timelines
market-index (5000000 followers): PULL at read time
Rajesh's feed: precomputed + merge celebrity cache
= Fast reads + no celebrity write storm
What supporting details make this production-grade?
Cache celebrity posts. Cap timelines. Async fanout. Dedup.
package com.bank.feed;
public class ProductionDetails {
public static void main(String[] args) {
System.out.println("Production details:");
System.out.println(" 1. Cache celebrity recent posts (many followers pull same posts)");
System.out.println(" 2. Cap timeline size to 500 entries (rarely scrolled far)");
System.out.println(" 3. Async fanout via queue (posting returns immediately)");
System.out.println(" 4. Denormalized entries (no joins at read time)");
System.out.println(" 5. Deduplication on merge");
System.out.println(" 6. Celebrity threshold from real follower distribution");
}
}
Expected output:
Production details:
1. Cache celebrity posts (many followers pull same)
2. Cap timelines to 500 (bound storage)
3. Async fanout (posting returns fast)
4. Denormalized entries (no joins)
5. Dedup on merge
6. Threshold from real data
How do you handle edge cases?
New follow backfill, deleted posts, celebrity threshold crossing.
package com.bank.feed;
public class EdgeCases {
public static void main(String[] args) {
System.out.println("Edge cases:");
System.out.println(" New follow: backfill last N posts into timeline");
System.out.println(" Deleted post: store references, check visibility at read");
System.out.println(" Crossing threshold: user gains 10K followers -> switch to pull");
System.out.println(" Inactive users: fanout wastes effort -> skip or lazy-load");
}
}
Expected output:
Edge cases:
New follow: backfill last N posts
Deleted post: check visibility at read
Crossing threshold: switch strategy
Inactive users: skip or lazy-load
How does the approach compare?
| Strategy | Write Cost | Read Cost | Best For |
|---|---|---|---|
| Fanout-on-write | O(followers) | O(1) | Normal users |
| Fanout-on-read | O(1) | O(followees) | Celebrities |
| Hybrid | Mixed | Fast for both | Production systems |
The hybrid exists to pay the cost where it is cheapest for each class of user.
Normal: cheap push (few followers), instant read
Celebrity: cheap store (one write), merge at read with caching
What should a senior engineer know?
The celebrity threshold trades write cost against read cost. Revisit it from the actual follower distribution. Store denormalized entries but reference canonical posts for edits and deletes.
Senior guidelines:
[x] Hybrid: push for normal, pull for celebrity
[x] Async fanout via queue
[x] Celebrity threshold from real follower distribution
[x] Cache celebrity posts
[x] Cap timelines (bound storage and latency)
[x] Denormalized entries for fast reads
[x] Handle: new follows, deletes, threshold crossing
Visual Overview
Hybrid Feed Design:
Normal User Posts (PUSH):
┌──────────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ analyst- │───▶│ Queue │───▶│ Worker │───▶│ Timeline │
│ pankaj posts │ │ (async) │ │ (fanout) │ │ follower │
│ │ └──────────┘ │ to 500 │ │ mailboxes│
└──────────────┘ │ timelines│ └──────────┘
└──────────┘
Celebrity Posts (PULL at read):
┌──────────────┐ ┌──────────┐
│ market-index │───▶│ Post │ (1 write, no fanout)
│ posts │ │ Store │
└──────────────┘ └─────┬────┘
│ on read
┌──────────────┐ │
│ Rajesh reads │ │
│ feed │ │
└──────┬───────┘ │
│ │
┌──────▼──────────────────▼──────┐
│ Merge: │
│ Precomputed timeline │
│ + Cached celebrity posts │
│ -> Sorted by time │
└───────────────────────────────┘
Key Takeaway
A social feed at scale is a trade-off about where you pay the cost. Fanout-on-write precomputes timelines for instant reads but triggers N writes per post where N is the follower count. Fanout-on-read stores posts once but makes every read an expensive multi-account merge. Neither pure strategy works because the follower distribution is extremely skewed: most users have few followers but some have millions.
The hybrid resolves this by choosing the strategy per author. Normal users get push: their posts are written into followers' timelines for instant reads. Celebrities get pull: their posts are stored once and merged at read time from a cache. Rajesh's feed reads his precomputed timeline plus a few cached celebrity pulls.
At National Bank of India, Pankaj built an investor-updates feed this way. Analysts with modest followings were fanned out on write. A market-index account with millions of followers was pulled at read time. The celebrity threshold, set from the real follower distribution, is the knob that trades write cost against read cost.
Remember one sentence: use fanout-on-write for normal users so reads are instant, and fanout-on-read for celebrities so one post never triggers millions of writes, giving a hybrid that pays the cost where it is cheapest for each class of user.
---