Media & streaming, an interactive architecture walkthrough
Video sharing platform (YouTube-style): how the architecture works
How a video platform takes resumable uploads, encodes segments in parallel into several renditions, and serves them via a CDN with adaptive bitrate.
This is a learning explainer based on public engineering writing and general design practice. It is simplified, it is not the actual architecture of any company, and it may be inaccurate.
Interactive diagram
Overview
A video sharing platform accepts large files from creators, turns each into many playable versions, and serves them to viewers on very different connections. The upload path writes the original in resumable chunks and emits an event. A processing pipeline splits the video into segments, encodes each segment into several resolutions and bitrates in parallel, and packages them behind a manifest. Viewers fetch the manifest and segments through a CDN, and the player switches quality as bandwidth changes.
Trade-offs and what changes at scale
Encoding is the main compute cost and encoding every rendition for every upload is wasteful, because most videos are rarely watched; many platforms encode a core ladder first and add expensive renditions or codecs only for videos that become popular. Splitting into segments cuts latency but adds stitching and consistency checks, and some processing (such as whole-video classification) is harder to run per segment. Retries make workers' output repeatable but cost duplicate compute. CDN egress and storage usually dominate the bill at scale, so cache hit ratio and rendition choice are cost decisions as well as quality ones. Recommendations, search, comments and monetisation are separate systems left out here. This is a simplified teaching model based on public information, not YouTube's actual architecture.
How it works, step by step
The walkthrough above, as text. Each step lights up the components named under it.
A creator starts an upload. The gateway authenticates them and applies upload quotas and rate limits, since an upload is expensive to store and encode.
The Upload Service creates an upload session and a metadata record in the Video Metadata DB with state 'uploading'. The session lets the client send the file in chunks and resume after a dropped connection.
Chunks are appended to Raw Upload Storage as they arrive. After an interruption the client asks how many bytes were stored and continues from there. Sessions can expire, so a long-abandoned upload starts again from the beginning.
When the last chunk is stored, the Upload Service publishes an upload-complete event. Processing starts from this event, so the upload path does no encoding work and stays responsive.
The Transcode Orchestrator consumes the event and plans the job: it splits the video into segments that can be encoded independently (aligned to groups of pictures) and decides which renditions (resolution and bitrate pairs) to produce.
Before the expensive encoding, the Checks Service verifies the file is a real video and runs policy and copyright checks. Its result is recorded on the video, so a video that fails can be blocked or held for review.
The orchestrator enqueues one task per segment per rendition. Priorities keep a newly uploaded video's tasks ahead of background work such as re-encoding old videos.
Transcode Workers pull tasks, read their source segment from Raw Upload Storage and encode it. Many workers run in parallel, which shortens a long encode considerably. Tasks are written to be repeatable, so a failed task can be retried on another worker.
Each worker writes its encoded segment to Media Storage under a name derived from the video, rendition and segment number, so a retry overwrites the same object rather than creating a duplicate. Workers report progress back to the orchestrator.
When the required tasks are complete, the orchestrator calls the Packager, which writes the manifests that list every rendition and its segments, plus thumbnails, into Media Storage and marks the video ready in the metadata DB. Some platforms make a video watchable once the first renditions are ready and add the rest later.
A viewer opens the video. The Playback Service reads its metadata, checks access (private, region or age restrictions) and returns the manifest location.
The player fetches the manifest and then segments through the CDN, picking a rendition per segment from measured throughput and buffer level, so quality moves up or down with the connection. Popular segments are served from edge caches.
The player sends view beacons. Analytics Ingest filters duplicates and suspected fake views and increments counters in the View Counts Store. Public view counts are therefore approximate and can lag the true figure.
Observability tracks queue depth, time from upload to watchable, encode failure rate, CDN hit ratio and playback rebuffering. Upload-to-ready time is the number creators notice first.
Sequence diagram
The same walkthrough as a UML sequence diagram, one lane per layer or group (18 components in all), left to right in the order a request reaches them, numbered by step. A loop on a lane is a step that stays inside that group. Solid arrows are synchronous calls, dashed are asynchronous.
Key components
Edgelayer
- API Gateway: TLS, auth, rate limiting
- CDN: caches manifests and video segments
Upload Pathdomain
- Raw Upload Storage: original files, kept for re-encoding (s3)
- Upload Events: upload-complete events (kafka)
- Upload Service: resumable sessions, chunk writes
- Video Metadata DB: title, owner, state, rendition list (postgresql)
Processing Pipelinedomain
- Checks Service: file validation, policy and copyright checks
- Media Storage: encoded segments, manifests, thumbnails (s3)
- Packager: stitches output, writes manifests, thumbnails
- Task Queue: encode tasks, high and low priority
- Transcode Orchestrator: plans segments and renditions, tracks progress
- Transcode Workers: encode one segment into one rendition
Playback and Analyticsdomain
- Analytics Ingest: view beacons, fraud and duplicate filtering
- Playback Service: access checks, manifest location
- View Counts Store: counters, eventually consistent
Observability
- Observability: queue depth, encode time, rebuffering
Clientslayer
- Uploader Client: web and mobile, resumable chunked upload
- Viewer Player: adaptive bitrate player
Sources
Public writing this explainer draws on. Read these for what the real systems actually do.
Make it yours
Copy this diagram and its walkthrough into your own workspace, then change anything: swap Postgres for DynamoDB, add a cache, rewrite the narration. Free, and it uses no AI credits.
Related walkthroughs
- Photo sharing platformHow a photo sharing platform takes uploads straight to object storage, resizes them in the background, and builds follower feeds from cached post ids.
- Live streaming platformHow a live streaming platform ingests one creator stream, transcodes it in real time, and serves short HLS segments to large audiences via a CDN.
- Netflix-style video streamingHow a streaming service encodes each title into many bitrates, caches it near viewers on a CDN, and personalises what each person sees.
This is a learning explainer based on public engineering writing and general design practice. It is simplified, it is not the actual architecture of any company, and it may be inaccurate. Real systems differ in the details. Browse the library.