Design the home timeline for a Twitter-scale social network.
Probes fanout design, hot-user handling, ranking, and caching — quintessential senior system design.
Two core approaches: push (fanout-on-write: when A tweets, write to each follower's timeline) vs pull (fanout-on-read: merge latest tweets from all follows at read time).
Push: read is cheap (one sequential read); write amplifies by follower count. Great for small-follower users, disastrous for celebrities.
Pull: read is expensive (merge N feeds); write is cheap. Works for low-read-frequency but doesn't scale to high-QPS reads.
Hybrid: push for normal users, pull for celebrities' posts (fetched at read time and merged). Industry standard.
Ranking: chronological → ML-scored. Feature store feeds a model that orders candidates. Keep last N hours eligible.
Push alone crashes on celebrities — one tweet by someone with 50M followers means 50M writes. Pull alone crashes on high read QPS — merging N feeds per read is too slow. The answer is hybrid: for most users push into their timeline (precomputed), but tag some users as 'celebrity' and fan-out only a limited snapshot; at read time, merge their latest tweets into the precomputed timeline. For ranking I'd add a feature store and an ML ranker — candidates from recent push writes + celebrity pulls go through a model scoring engagement likelihood. Store media on object storage (S3) with CDN; timeline entries reference IDs, not content. Caching: Redis for hot timelines, TTL minutes, write-through on new post. The interesting variants are 'what if someone unfollows' (lazy cleanup on next read) and 'what about edits/deletes' (mark tombstones, filter at read).
Precomputed timelines are throughput-optimized; real-time ranking adds latency — pick your spot.
Celebrity threshold (e.g. 100K followers) is tuned empirically from storage cost vs latency curves.
HireMe AI (即答侠) walks through the "celebrity problem" with real numbers so interviewers see you've grokked the trade-off.
Tombstone in the write path, filter at read. Garbage-collect tombstones periodically once they fall out of the active window.
Chronological is easier to debug and users sometimes demand it. Algorithmic wins on engagement metrics. Most platforms offer both modes.
HireMe AI listens to your interview in real-time, detects questions, and generates instant answer suggestions — so you never freeze up again.
Try HireMe AI Free