Web fundamentals, an interactive architecture walkthrough
Load balancer: how the architecture works
How a two-tier load balancer spreads traffic with ECMP and consistent hashing at L4, then balances each request at L7 using health checks.
This is a learning explainer based on public engineering writing and general design practice. It is simplified, it is not the actual architecture of any company, and it may be inaccurate.
Interactive diagram
Overview
A load balancer spreads incoming connections and requests across many servers so no single server is overloaded and a failed server stops receiving traffic. Large deployments use two tiers: a layer 4 tier that forwards packets by hashing the connection's addresses and ports, and a layer 7 tier of proxies that terminate TLS and choose a backend for each HTTP request.
The L4 tier is kept stateless with consistent hashing, so routers can send a flow to any L4 node and still reach the same proxy, and replies skip the L4 tier entirely. The L7 tier applies per-request algorithms such as least-request, removes unhealthy instances, and drains instances during deploys. A control plane combines health checks and service discovery and pushes the result to both tiers. This is a simplified learning design, not any provider's actual implementation.
Trade-offs and what changes at scale
Round robin is simple but ignores how busy a server is; least-request needs per-request visibility, which only an L7 proxy has. Consistent hashing keeps most flows in place when the set changes, but some still move, which is why L4 nodes also keep a flow table. Sticky sessions are easy but unbalance load, so session state is moved to a shared store. Terminating TLS at the proxy enables routing on content but means re-encrypting to backends where the network is not trusted. At global scale DNS steers users between regions while anycast or ECMP spreads traffic inside each one.
How it works, step by step
The walkthrough above, as text. Each step lights up the components named under it.
A client resolves the site's name and GeoDNS answers with the virtual IP of the nearest healthy region, using a short TTL. DNS spreads load across regions, but clients cache answers, so it cannot react within seconds.
Packets for the VIP reach the Edge Routers, where every L4 node announces the same address, and equal-cost multipath hashing spreads flows across them after the DDoS Filter. Adding an L4 node is a route announcement, not a DNS change.
Each of the L4 Load Balancer Nodes hashes the connection's addresses and ports with a consistent hash such as Maglev to pick an L7 proxy. Because the choice depends only on packet headers, any node picks the same proxy for the same flow.
The chosen proxy is remembered in the node's Flow Table, so a live connection keeps its proxy even when the proxy set changes. Without it, a resize would move some in-flight connections and reset them.
The L4 node encapsulates the packet to one of the L7 Proxies, and replies go straight back to the client (direct server return). The L4 tier only sees the small inbound packets, so a few nodes can carry a lot of traffic.
The L7 Proxies terminate TLS with certificates issued and rotated by the Certificate Manager. Decrypting here lets the proxy route on host, path and headers.
For the API Pool the proxy uses least-request with two random choices: it samples two instances and sends the request to the less busy one. This avoids herding onto one server, which plain round robin can do when requests vary in cost.
WebSocket upgrades go to the WebSocket Pool and are balanced by open connection count, since one connection can last hours. Per-request algorithms would not see that load.
Backends keep sessions in the Session Store instead of in memory, so any instance can serve any request. That removes the need for sticky sessions, which unbalance load and lose sessions when a server dies.
The Health Checker probes each instance and reports changes to the LB Controller, and proxies also eject instances that return repeated errors. Active probes catch dead servers; passive checks catch servers that answer probes but fail real requests.
The LB Controller combines health results with instances from Service Discovery, including ones the Autoscaler just added, and pushes new pools to the proxies. During a deploy an instance is marked draining so it finishes current requests before it is removed.
Proxies write one record per request to Access Logs, which load into Log Analytics, and send live metrics to Observability. Per-pool latency and error rates show whether the balancing is actually even.
Sequence diagram
The same walkthrough as a UML sequence diagram, one lane per layer or group (19 components in all), left to right in the order a request reaches them, numbered by step. A loop on a lane is a step that stays inside that group. Solid arrows are synchronous calls, dashed are asynchronous.
Key components
Clientslayer
- Browsers & Apps: resolve the name, open connections
- GeoDNS: returns the nearest healthy region's VIP
Network Edgelayer
- DDoS Filter: drops floods and malformed packets
- Edge Routers: ECMP across L4 nodes announcing the VIP
L4 Tierlayer
stateless packet forwarding
- Flow Table: per-node map of live flows to backends
- L4 Load Balancer Nodes: consistent hash on the 5-tuple
L7 Tierlayer
- Certificate Manager: issues and rotates TLS certificates
- L7 Proxies: TLS, HTTP routing, per-request balancing
Backend Poolslayer
- API Pool: JSON and gRPC APIs (grpc)
- Session Store: sessions kept off the servers (redis)
- Web Pool: server-rendered pages
- WebSocket Pool: long-lived connections (websocket)
Control Planedomain
- Autoscaler: adds instances on load
- Health Checker: active HTTP/TCP probes
- LB Controller: pools, weights, drains, VIPs
- Service Discovery: registered backend instances
Telemetrylayer
- Access Logs: per-request records (kafka)
- Log Analytics: latency and errors by pool (clickhouse)
- Observability: connection counts, health, alerts
Sources
Public writing this explainer draws on. Read these for what the real systems actually do.
Make it yours
Copy this diagram and its walkthrough into your own workspace, then change anything: swap Postgres for DynamoDB, add a cache, rewrite the narration. Free, and it uses no AI credits.
Related walkthroughs
- Distributed cacheHow a distributed cache places keys with consistent hashing, replicates shards, evicts entries, invalidates on writes and resists stampedes.
- Distributed key-value storeHow a Dynamo-style key-value store places keys on a hash ring, replicates them with tunable quorums, and repairs replicas that fall behind.
- URL shortenerHow a link shortener creates unique short keys, serves redirects from a cache in a few milliseconds, and counts clicks off the redirect path.
This is a learning explainer based on public engineering writing and general design practice. It is simplified, it is not the actual architecture of any company, and it may be inaccurate. Real systems differ in the details. Browse the library.