AI applications, an interactive architecture walkthrough
Recommendation system: how the architecture works
How a recommendation system retrieves thousands of candidates from millions of items, ranks them with a model, and filters for freshness and diversity.
This is a learning explainer based on public engineering writing and general design practice. It is simplified, it is not the actual architecture of any company, and it may be inaccurate.
Interactive diagram
Overview
A recommendation system picks, for each user and moment, a handful of items from a catalog of millions. It works as a funnel: cheap retrievers such as nearest-neighbour search over learned embeddings, co-viewed items and trending lists select a few thousand candidates, then a heavier ranking model scores those candidates using user, item and context features, and business rules filter and diversify the final list.
Interaction events feed both paths. A streaming job keeps recent-activity features fresh in an online feature store, while daily training jobs rebuild retrieval and ranking models from history, using features as they were at the time of each event. New models ship only after an online experiment shows they improve engagement. This is a simplified learning design, not any company's production system.
Trade-offs and what changes at scale
More candidates give the ranker more to choose from but cost latency, so the funnel sizes are tuned together. Optimising for clicks alone tends to favour sensational or repetitive items, which is why diversity rules and longer-term metrics are used. New users and new items have little data, so popularity and content-based retrievers fill the gap. At larger scale the ANN index is sharded and refreshed incrementally, and feature lookups are batched because they dominate serving time.
How it works, step by step
The walkthrough above, as text. Each step lights up the components named under it.
User Apps send impressions and actions such as clicks, purchases and skips to the Event Collector, which publishes them to Interaction Events. Impressions matter as much as clicks, because an item shown and ignored is a negative example.
Streaming Features turns the last few minutes of activity into features such as categories viewed this session and writes them to the Online Feature Store. Recent behaviour is often the strongest signal of what someone wants next.
Each day the Training Pipeline builds training sets from event history and item metadata and trains a retrieval model and a ranking model. Features are taken as they were at the time of each event, so the model never learns from information it would not have had.
New models are stored in the Model Registry with their offline metrics. A model only goes to production if it beats the current one on held-out data and then in an online experiment.
The Embedding Builder computes a vector for every item into the ANN Index and a vector for every user into the feature store. Items close to a user's vector are likely matches, and the index finds them without scanning all items.
When the home page loads, the Recommendation API asks the Experiment Service which model variant this user is in. Every request is tagged with that variant so results can be compared later.
Candidate Sources queries several retrievers in parallel: nearest items to the user's vector, items co-viewed with recent ones and trending items from Popular Items. Mixing sources gives new users and new items a chance even when embeddings know little about them.
The API hydrates each of the few thousand candidates with user, item and context features from the Online Feature Store. Ranking needs the same features the model was trained on, read in milliseconds.
The Ranker predicts how likely the user is to engage with each candidate and sorts them. This is the expensive model, affordable only because it scores thousands of items instead of millions.
Filters & Diversity remove items the user has seen, items out of stock and near-duplicates, and spread categories so one topic does not fill the page. These rules encode product decisions the model is not trained to make.
Impressions and actions load into the Metrics Store, where click-through, conversion and long-term engagement are compared per variant. Offline accuracy does not always predict online results, so the experiment decides.
Sequence diagram
The same walkthrough as a UML sequence diagram, one lane per layer or group (18 components in all), left to right in the order a request reaches them, numbered by step. A loop on a lane is a step that stays inside that group. Solid arrows are synchronous calls, dashed are asynchronous.
Key components
Signalslayer
- Catalog DB: items, categories, availability (postgresql)
- Event Collector: validates and batches events
- Interaction Events: impressions and actions (kafka)
- User Apps: views, clicks, purchases, skips
Featuresdomain
- Online Feature Store: user and item features by key (redis)
- Streaming Features: last 30 minutes of activity (flink)
Offline Trainingdomain
- Data Lake: event history and training sets (s3)
- Embedding Builder: user and item vectors
- Model Registry: versioned models and metrics (s3)
- Training Pipeline: retrieval and ranking models, daily
Online Servingdomain
- ANN Index: approximate nearest-neighbour item vectors
- Candidate Sources: fan out to retrievers
- Filters & Diversity: seen, unavailable, too similar
- Popular Items: trending lists per region (redis)
- Ranker: predicts engagement per candidate
- Recommendation API: home feed, related items, emails
Experimentsdomain
- Experiment Service: assigns users to model variants
- Metrics Store: CTR, conversion, watch time by variant (clickhouse)
Sources
Public writing this explainer draws on. Read these for what the real systems actually do.
- twitter/the-algorithm: Home Mixer pipeline (candidates, feature hydration, ranking, filtering)
- microsoft/recommenders: best practices for building and evaluating recommenders
- facebookresearch/faiss: similarity search over dense vectors
- feast-dev/feast: online and offline feature stores, point-in-time correctness
Make it yours
Copy this diagram and its walkthrough into your own workspace, then change anything: swap Postgres for DynamoDB, add a cache, rewrite the narration. Free, and it uses no AI credits.
Related walkthroughs
- RAG chatbot over your docsHow a retrieval-augmented chatbot ingests company documents, retrieves permitted passages with hybrid search and reranking, and answers with citations.
- Semantic search with embeddingsHow semantic search embeds passages into a vector index, runs keyword search alongside it, fuses both rankings, and reranks the top results.
- ML feature storeHow a feature store computes features once, keeps timestamped history for point-in-time training sets, and serves the latest values to live models.
This is a learning explainer based on public engineering writing and general design practice. It is simplified, it is not the actual architecture of any company, and it may be inaccurate. Real systems differ in the details. Browse the library.