Design a News Feed / Timeline (like Twitter/Instagram)

• Fan-out-on-write : when someone posts, immediately write the post id into every follower's precomputed feed list.
• Fan-out-on-read : assemble the feed on demand by pulling recent posts from everyone the user follows.
• Celebrity problem : a user with millions of followers makes fan-out-on-write prohibitively expensive.
• Hybrid : fan-out-on-write for normal accounts, pull-on-read for celebrities, merged at read time.
• Cursor pagination : use the last-seen post id or score as the cursor, stable when new posts arrive.

What is fan-out-on-write?

When someone posts, the system immediately writes that post id into every follower's feed list in a fast store like Redis. Reading is trivial: just fetch the precomputed list. At National Bank of India, when an advisor posts a market update, it is pushed into the feed of every customer who follows that advisor.

package com.bank.feed;

import java.util.List;

public class FanOutOnWrite {
    public void onNewPost(String authorId, String postId) {
        List<String> followers = followerStore.of(authorId);
        for (String follower : followers)
            feedCache.prepend(follower, postId);  // Redis LPUSH per follower
    }
}

What is fan-out-on-read?

Nothing is precomputed. When Rajesh opens the feed, the system queries recent posts from every account he follows and merges them by rank. Writes are cheap but reads are expensive because the merge happens on every request.

package com.bank.feed;

import java.util.ArrayList;
import java.util.List;

public class FanOutOnRead {
    public List<String> buildFeed(String userId, int limit) {
        List<String> followed = followStore.followedBy(userId);
        List<String> merged = new ArrayList<>();
        for (String author : followed)
            merged.addAll(postStore.recent(author, limit));
        merged.sort(ranking.byScoreDesc());
        return merged.subList(0, Math.min(limit, merged.size()));
    }
}

What is the celebrity problem?

A hugely popular account posting fans out to millions of feeds, which is wasteful and slow. One post from a mega-channel like the bank's official announcements would trigger millions of Redis writes. The hybrid solution uses fan-out-on-write for normal accounts and fan-out-on-read for celebrities, merged at read time.

package com.bank.feed;

import java.util.ArrayList;
import java.util.List;

public class HybridFeed {
    public List<String> buildFeed(String userId, int limit) {
        List<String> feed = new ArrayList<>(feedCache.range(userId, 0, limit));
        for (String celeb : celebritiesFollowedBy(userId))
            feed.addAll(postStore.recent(celeb, limit));  // pull on read
        feed.sort(ranking.byScoreDesc());
        return feed.subList(0, Math.min(limit, feed.size()));
    }
}

How does ranking work?

Ranking replaces pure chronological order with a score combining recency, engagement, and affinity. Recency decays over time. Engagement counts likes, shares, and comments. Affinity measures how often the reader interacts with the author. The ranking formula is tuned by the product team and must be updatable without rebuilding all feeds.

package com.bank.feed;

public class RankingService {
    public double score(Post post, String readerId) {
        double recency = 1.0 / (1.0 + hoursAgo(post.createdAt()));
        double engagement = post.likes() * 1.0 + post.shares() * 3.0 + post.comments() * 2.0;
        double affinity = interactionScore(readerId, post.authorId());
        return recency * 0.4 + engagement * 0.3 + affinity * 0.3;
    }
}

Why use cursor pagination?

A new post arriving between page loads shifts offset-based pages, causing duplicates or gaps. A cursor based on the last-seen post id or score is stable because it anchors to a specific position in the result set regardless of new inserts.

package com.bank.feed;

import java.util.List;

public class CursorPaginator {
    public List<String> nextPage(String userId, String lastSeenPostId, int pageSize) {
        double lastScore = feedCache.scoreOf(userId, lastSeenPostId);
        return feedCache.rangeByScore(userId, lastScore, pageSize);
    }
}

How do you cache feeds?

Cache assembled feeds in Redis with a short TTL (e.g., 60 seconds) and invalidate or append on new posts. Accept eventual consistency: a feed being seconds stale is fine because unlike a bank balance, a slightly outdated feed harms nobody.

How is this different from a chat system?

A chat requires strong ordering and delivery guarantees per conversation. A feed is a ranked list where eventual consistency and approximate ordering are acceptable. Chat messages must never be lost; a feed post appearing a few seconds late is fine.

What should a senior engineer know?

A senior engineer tunes the celebrity threshold (e.g., 100K followers), monitors fan-out latency per percentile, and designs the ranking algorithm to be updatable via configuration without rebuilding all feeds. They also consider A/B testing different ranking formulas and monitoring feed freshness metrics across regions.