Web fundamentals, an interactive architecture walkthrough
Pastebin: how the architecture works
How a pastebin stores text in object storage behind a small metadata table, serves reads from a CDN and cache, and expires pastes on time.
This is a learning explainer based on public engineering writing and general design practice. It is simplified, it is not the actual architecture of any company, and it may be inaccurate.
Interactive diagram
Overview
A pastebin stores a block of text and returns a short URL that anyone with the link can open. Writes are rare compared with reads, and a paste never changes after it is created, so the design splits a small metadata record in a database from the body in object storage, and serves public pastes through a CDN and a cache.
Keys are random, so unlisted pastes cannot be discovered by counting through URLs. Expiry is enforced in two places: the read path checks the expiry time on every request, and a background worker deletes expired records and objects to reclaim storage. Highlighting, abuse scanning and view counting run from an event stream instead of on the request. This is a simplified learning design, not any specific site's architecture.
Trade-offs and what changes at scale
Random keys avoid enumeration but need a collision check, which a conditional insert makes cheap; counter-based keys never collide but are guessable unless scrambled. Long CDN caching is safe because pastes are immutable, but takedowns and early deletes then need an explicit purge. Pre-rendering highlighted HTML doubles storage per paste in exchange for cheaper reads. At larger scale storage, not traffic, is the main cost, so default expiry times, compression and cheaper storage tiers for old pastes matter more than adding read servers.
How it works, step by step
The walkthrough above, as text. Each step lights up the components named under it.
A paste arrives from the browser or a CLI / API Client at the API Gateway, which enforces a maximum size and per-IP or per-token rate limits. Anonymous writes are the main abuse risk, so these limits sit at the edge.
The Paste Service asks the Key Generator for a random 8-character base62 key. Random keys cannot be enumerated, which matters for unlisted pastes that rely on nobody guessing the URL.
The paste body is written to the Content Store as an object, while only a pointer to it goes in the database. Bodies can be megabytes, and object storage is cheaper and scales independently of the metadata.
The key, owner, visibility, expiry time and object path are written to the Metadata DB with an insert that fails if the key already exists. On the rare collision the service picks a new key and retries.
A paste-created event lets the Syntax Highlighter render the HTML once and store it next to the raw body. Rendering at write time means a popular paste is not re-highlighted on every view.
The Abuse Scanner checks new pastes for spam, malware links and leaked credentials, and sends hits to the Moderation Console, where a reviewer can take a paste down. Scanning asynchronously keeps posting fast; a takedown also purges the CDN copy.
Public pastes are read through the CDN, which serves the cached page until the paste's expiry. A paste's content never changes after it is created, so it can be cached safely for as long as it lives.
On a CDN miss, the Read Service looks up the key in the Paste Cache, then the Metadata DB, and checks the expiry time before fetching the body. Expired or deleted pastes return 404 even if a background cleanup has not run yet.
Entries in the Paste Cache carry a TTL no later than the paste's expiry, so the cache cannot outlive the paste. Private and burn-after-reading pastes skip the CDN entirely and are marked no-store.
Each view publishes an event that the View Counter aggregates into the Analytics Store. Counting off the read path keeps reads fast, and counts lag by a few seconds.
The Expiry Worker periodically finds expired pastes in the Metadata DB, deletes their objects and evicts them from the cache. This reclaims storage, while the read-time expiry check is what makes a paste stop working on time.
Observability tracks read and write latency, CDN hit ratio and storage growth. Storage keeps growing for pastes that never expire, so that trend drives capacity planning.
Sequence diagram
The same walkthrough as a UML sequence diagram, one lane per layer or group (19 components in all), left to right in the order a request reaches them, numbered by step. A loop on a lane is a step that stays inside that group. Solid arrows are synchronous calls, dashed are asynchronous.
Key components
Edgelayer
- API Gateway: TLS, size limits, rate limits
- Auth Service: optional accounts and API tokens
- CDN: caches public pastes
Write Pathdomain
- Key Generator: random 8-char base62 keys
- Paste Service: validate, assign key, store
- Syntax Highlighter: pre-renders highlighted HTML
Storagelayer
- Content Store: paste bodies and rendered HTML (s3)
- Metadata DB: key, owner, expiry, object path (postgresql)
Backgroundlayer
- Abuse Scanner: spam, malware links, leaked secrets
- Analytics Store: views by paste and day (clickhouse)
- Expiry Worker: deletes expired pastes
- Moderation Console: review reports, take down
- Paste Events: created, viewed, reported (kafka)
- View Counter: aggregates views per paste
Read Pathdomain
- Paste Cache: hot metadata and small bodies (redis)
- Read Service: resolve key, check expiry, serve
Observability
- Observability: latency, error rates, storage growth
Clientslayer
- Browser: paste form and viewer
- CLI / API Client: curl, editor plugins
Sources
Public writing this explainer draws on. Read these for what the real systems actually do.
Make it yours
Copy this diagram and its walkthrough into your own workspace, then change anything: swap Postgres for DynamoDB, add a cache, rewrite the narration. Free, and it uses no AI credits.
Related walkthroughs
- URL shortenerHow a link shortener creates unique short keys, serves redirects from a cache in a few milliseconds, and counts clicks off the redirect path.
- Distributed cacheHow a distributed cache places keys with consistent hashing, replicates shards, evicts entries, invalidates on writes and resists stampedes.
- Search autocomplete (typeahead)How search autocomplete answers in milliseconds: prefix indexes, caching, offline ranking from query logs, trending queries and policy filtering.
This is a learning explainer based on public engineering writing and general design practice. It is simplified, it is not the actual architecture of any company, and it may be inaccurate. Real systems differ in the details. Browse the library.