Web fundamentals, an interactive architecture walkthrough
URL shortener: how the architecture works
How a link shortener creates unique short keys, serves redirects from a cache in a few milliseconds, and counts clicks off the redirect path.
This is a learning explainer based on public engineering writing and general design practice. It is simplified, it is not the actual architecture of any company, and it may be inaccurate.
Interactive diagram
Overview
A URL shortener maps a short key to a long URL. Reads far outnumber writes, so the redirect path is built around a cache in front of a sharded store, while key generation avoids collisions by leasing counter ranges to each server. Click events leave the redirect path through a queue, so a slow or unavailable analytics pipeline does not block redirects (the publish is fire-and-forget with a bounded buffer).
Trade-offs and what changes at scale
Using a 302 keeps every click visible to the service and so countable, at the cost of more redirect traffic than a cacheable 301. Counter ranges avoid collisions but make keys sequential, so they are scrambled before encoding; a pre-generated key pool or hashing are alternatives with their own collision handling. At large scale the work is mostly reads: add redirect replicas and cache capacity first, spread hot links across cache replicas, and reshard the mapping store only when storage requires it. Link ownership and accounts (needed for the dashboard) live in a small users/links table, omitted from the diagram for clarity. Click counts are eventually consistent, and a few can be lost if the event pipeline drops messages.
How it works, step by step
The walkthrough above, as text. Each step lights up the components named under it.
A user submits a long URL. The gateway terminates TLS and applies per-client rate limits, which limits how fast any one client can consume keys or write capacity.
The create service validates the URL (scheme, length, blocklist) and handles any custom alias or expiry before spending a key.
The key service hands out unique keys from a counter range leased from ZooKeeper: no per-request coordination and no collisions across nodes. The counter value is scrambled before base62 encoding, so links are short and hard to guess or enumerate (a keyed, reversible scramble of the counter). Links are still not secrets: do not use them as access control. A crash loses at most the unused part of a leased range.
The mapping is written to the store, which is sharded by a hash of the short key so writes spread evenly. The cache can be warmed up front for links expected to be popular; otherwise it fills on the first read, which avoids spending memory on links nobody clicks. Custom aliases are written with a conditional insert that fails if the alias is already taken.
A click on the short link reaches the redirect service through the load balancer. Redis is checked first; on a hit the service answers with a 302 redirect, typically within a few milliseconds. A 302 (not a permanent 301) keeps every click reaching us so it can be counted; a 301 would be cached by browsers and hide repeat clicks, but would save load.
On a cache miss the redirect service reads the shard directly, then repopulates Redis. Unknown or expired keys return 404 or 410; unknown keys can be cached briefly (negative caching) so repeated misses do not hit the store.
Each redirect publishes a click event asynchronously, without waiting for the result, so a slow or unavailable analytics pipeline does not block the redirect (the publish is fire-and-forget with a bounded buffer). If Kafka is down, redirects still work; a few click events may be lost, which is acceptable for analytics.
The stream processor aggregates clicks by link, referrer and geography in time windows, then writes to ClickHouse, which is built for columnar aggregation.
Dashboard reads go through the gateway to the analytics API, which queries ClickHouse and never touches the primary store.
The expiry worker periodically deletes expired links and evicts them from Redis, which frees keys and storage. The redirect service also checks the expiry time on every read and cache entries carry a TTL, so a link stops working on time even if the worker is late.
Observability tracks latency, hit ratio, consumer lag and key-range exhaustion across the services that report to it.
Sequence diagram
The same walkthrough as a UML sequence diagram, one lane per layer or group (14 components in all), left to right in the order a request reaches them, numbered by step. A loop on a lane is a step that stays inside that group. Solid arrows are synchronous calls, dashed are asynchronous.
Key components
Edgelayer
- Load Balancer / API Gateway: TLS, routing, rate limiting
Write Pathdomain
- Create Short URL Service: validate URL, assign key, store
- Key Generation Service: unique base62 keys from scrambled counter ranges
- ZooKeeper: allocates counter ranges (zookeeper)
Primary Datalayer
- URL Mapping Store: sharded by hash of short key
Read Pathdomain
- Redirect Service: resolve key, return 302 redirect
- URL Cache: hot short key to URL mappings (redis)
Analytics Pipelinelayer
- Analytics API: dashboard queries
- Analytics Store: click counts, referrers, geography (clickhouse)
- Click Events: redirect events topic (kafka)
- Stream Processor: aggregates clicks, referrers, geo
Expiry Worker
- Expiry Worker: deletes expired links, evicts cache
Observability
- Observability: metrics, alerts, dashboards
Clientslayer
- Mobile App: create and share links
- Web App: create links, analytics dashboard
Sources
Public writing this explainer draws on. Read these for what the real systems actually do.
Make it yours
Copy this diagram and its walkthrough into your own workspace, then change anything: swap Postgres for DynamoDB, add a cache, rewrite the narration. Free, and it uses no AI credits.
Related walkthroughs
- PastebinHow a pastebin stores text in object storage behind a small metadata table, serves reads from a CDN and cache, and expires pastes on time.
- Distributed cacheHow a distributed cache places keys with consistent hashing, replicates shards, evicts entries, invalidates on writes and resists stampedes.
- Search autocomplete (typeahead)How search autocomplete answers in milliseconds: prefix indexes, caching, offline ranking from query logs, trending queries and policy filtering.
This is a learning explainer based on public engineering writing and general design practice. It is simplified, it is not the actual architecture of any company, and it may be inaccurate. Real systems differ in the details. Browse the library.