Messaging & social, an interactive architecture walkthrough
Twitter-style timeline system: how the architecture works
How a timeline system fans tweets out to followers' cached timelines, pulls in large accounts at read time, and serves a home feed from cache.
This is a learning explainer based on public engineering writing and general design practice. It is simplified, it is not the actual architecture of any company, and it may be inaccurate.
Interactive diagram
Overview
A timeline system lets users post short messages and read a home timeline built from the accounts they follow. Reads far outnumber writes, so the usual design does the expensive work when a tweet is posted: a fan-out worker pushes the new tweet's id into a cached timeline for every follower, and reading a timeline becomes one cache lookup plus a hydration step. Tweets get time-ordered 64-bit ids, which lets timelines be sorted and paged by id.
Posting, fan-out and search indexing are decoupled by an event stream, so the user's post returns before followers' timelines are updated. A ranking step can reorder and filter the candidates; a strictly chronological timeline skips it. This is a simplified learning design, not any company's production system.
Trade-offs and what changes at scale
Fan-out on write makes reads cheap but multiplies writes by the follower count, so accounts with very large followings are handled with fan-out on read instead and merged in when a timeline is served. Timelines are eventually consistent: a follower may see a tweet a few seconds after it is posted. Caching timelines only for active users caps memory but means a returning user's first load is a slower rebuild from the stores. At larger scale the graph and tweet stores are sharded by user and tweet id, the timeline cache is replicated, and out-of-network recommendations add more candidate sources to the read path.
How it works, step by step
The walkthrough above, as text. Each step lights up the components named under it.
A user posts a tweet from the Mobile Client, and the API Gateway authenticates the request and applies a per-user rate limit. Limits sit at the edge so spam bursts never reach the write path.
Attached photos or video are uploaded first to the Media Service, which stores them in the Media Store and returns a media id. The tweet then only carries that id, so the tweet write stays small.
The Tweet Service validates the text and asks the ID Generator for a 64-bit id built from a timestamp, a worker number and a sequence. Because ids are roughly time-ordered, timelines can be sorted by id without a separate timestamp index.
The tweet is written to the Tweet Store, sharded by tweet id so writes spread evenly, and copied into the Tweet Cache. Most reads of a new tweet happen in its first hours, which is when the cache copy matters.
The Tweet Service publishes a tweet-created event to the Tweet Events topic in Kafka and returns to the user. Fan-out and indexing run behind the topic, so posting stays fast even when an author has many followers.
The Fan-out Service consumes the event and asks the Social Graph Service for the author's followers. This is fan-out on write: the cost is paid once per tweet instead of on every timeline read.
For each follower, the Fan-out Service pushes the tweet id onto that follower's list in the Timeline Cache, trimmed to a few hundred entries. Only recently active users get a cached list, which keeps Redis memory bounded.
When a follower opens the app, the Timeline Service reads their list of tweet ids from the Timeline Cache in one call. An inactive user has no list, so it is rebuilt from the Tweet Store and the graph on that first read.
The ids are hydrated with a multi-get against the Tweet Cache, falling back to the Tweet Store, and author profiles come from the User Service. Storing only ids in each timeline means an edited or deleted tweet changes in one place.
Accounts with millions of followers are skipped at fan-out time, so the Timeline Service asks the Social Graph Service which of them the reader follows and pulls their recent tweets at read time. This hybrid avoids minutes-long fan-out jobs for a single tweet.
The merged candidates go to the Ranking Service, which scores them and drops muted, blocked or already-seen tweets before the page is returned. A chronological timeline would skip this step and sort by id.
A Search Indexer consumes the same events and adds each tweet to the Search Index, and the Search Service queries every shard and merges the results. Indexing from the stream keeps search a few seconds behind posting without slowing it.
Clients load images and video from the Media CDN, which pulls from the Media Store on a miss. Media bytes never pass through the timeline services.
Sequence diagram
The same walkthrough as a UML sequence diagram, one lane per layer or group (21 components in all), left to right in the order a request reaches them, numbered by step. A loop on a lane is a step that stays inside that group. Solid arrows are synchronous calls, dashed are asynchronous.
Key components
Clientslayer
- Mobile Client: iOS and Android apps
- Web Client: browser timeline and composer
Edgelayer
- API Gateway: TLS, auth, per-user rate limits
- Media CDN: caches images and video near users
Write Pathdomain
- ID Generator: 64-bit time-ordered ids
- Media Service: upload, transcode, media ids
- Media Store: original and resized media (s3)
- Tweet Cache: hot tweet bodies by id (redis)
- Tweet Service: validate, assign id, store tweet
- Tweet Store: tweets sharded by tweet id (mysql)
Fan-outdomain
- Fan-out Service: push tweet ids to follower timelines
- Graph Store: follow edges, sharded by user (mysql)
- Social Graph Service: followers and following lists
- Tweet Events: tweet created / deleted topic (kafka)
Read Pathdomain
- Ranking Service: scores and filters candidates
- Timeline Cache: per-user list of recent tweet ids (redis)
- Timeline Service: builds the home timeline
- User Service: profiles, avatars, mute/block lists
Searchdomain
- Search Index: inverted index, recent tweets hot (elasticsearch)
- Search Indexer: tokenizes and indexes new tweets
- Search Service: parse query, scatter-gather, rank
Observability
- Observability: metrics, consumer lag, alerts
Sources
Public writing this explainer draws on. Read these for what the real systems actually do.
Make it yours
Copy this diagram and its walkthrough into your own workspace, then change anything: swap Postgres for DynamoDB, add a cache, rewrite the narration. Free, and it uses no AI credits.
Related walkthroughs
- Social news feed (Twitter-style timeline)How a social feed pushes posts into follower timelines, handles accounts with millions of followers by merging at read time, and ranks the result.
- WhatsApp-style chat appHow a chat system holds millions of persistent connections, routes end-to-end encrypted messages, and falls back to push when users are offline.
- Notification system at scaleHow a notification platform validates requests, applies user preferences, queues by priority, and delivers push, SMS and email with retries and fallbacks.
This is a learning explainer based on public engineering writing and general design practice. It is simplified, it is not the actual architecture of any company, and it may be inaccurate. Real systems differ in the details. Browse the library.