Multi-tenant SaaS analytics pipeline: how the architecture works

How it works, step by step

The walkthrough above, as text. Each step lights up the components named under it.

1. 01

   An end-user action in the tenant's app is captured by the SDK, which batches events and attaches the tenant's write key before sending. Write keys are public in browsers, so abuse is limited by origin checks and a per-key rate limit.

   Components Analytics SDK, End Users · Via app events

2. 02

   The gateway terminates TLS and routes the batch to the collector, shielding it from malformed or oversized traffic.

   Components API Gateway, Collector Service · Via HTTPS batch

3. 03

   The collector resolves the write key to a tenant from a short-lived cache of the control plane's data in Redis (refreshed or invalidated by the control plane when keys change), going to the control plane only on a miss so ingestion doesn't depend on it for every batch. It then applies per-tenant rate limits and quotas from Redis. Bad keys or exhausted quotas are rejected early.

   Components Collector Service, Control Plane API, Rate Limit Store · Via forward

4. 04

   Events are validated against the schema registry so events that fail validation are rejected before the stream and cannot poison downstream consumers. Free-form properties are validated loosely.

   Components Schema Registry · Via validate

5. 05

   Valid events are published to Kafka keyed by tenant and user or session id. That spreads one large tenant's load across partitions while each user's events stay in order, and the stream is a durable buffer if downstream slows.

   Components Event Stream · Via publish event

6. 06

   The stream processor enriches events with geo/IP and user agent, deduplicates and sessionises them. Keying by session keeps one user's state on one partition, so sessionisation stays correct.

   Components Stream Processor · Via consume

7. 07

   The stream processor writes processed events to ClickHouse in large batches (ClickHouse wants few big inserts, not many tiny ones), with tenant_id on every row; late or duplicate events are handled by deduplication within a time window. Raw copies are archived as Parquet in S3 for replay and reprocessing, and the archive can be read back into the stream processor.

   Components Analytics Store, Raw Event Archive · Via insert

8. 08

   A tenant analyst opens the dashboard and signs in; the identity provider issues a JWT carrying the tenant id.

   Components Dashboard Web App, Tenant Analyst, Identity Provider · Via opens

9. 09

   The query API verifies the JWT locally with the identity provider's cached signing keys and injects the tenant filter itself, so clients do not choose the tenant. This holds because the API accepts structured queries, not raw SQL. ClickHouse row policies or per-tenant views can enforce the same filter in the database as a second line of defence.

   Components Query API, Identity Provider · Via HTTPS + JWT

10. 10

    The tenant-keyed Redis cache is checked first; on a miss the scoped query runs on ClickHouse and the result is cached, protecting the store from repeated dashboards.

    Components Query Cache, Analytics Store · Via cache check

11. 11

    The collector counts usage in memory and Redis, never with a Postgres write per request. Redis counters are volatile, so they drive quota enforcement and an early estimate; the billable figure is reconciled against deduplicated event counts in ClickHouse. The billing job reads those counters on a schedule, aggregates them per tenant against plans, writes totals to Postgres, and reconciles against those ClickHouse counts before invoicing. Quota enforcement uses the same counters.

    Components Metering & Billing Job, Analytics Store, Rate Limit Store, Control DB · Via aggregate usage

12. 12

    A tenant admin onboards through the control plane, which creates the tenant, plan and write keys in its Postgres; keys and quotas then reach the collector through the Redis cache. Noisy-neighbour limits keep one tenant from starving others.

    Components Control Plane API, Control DB, Collector Service · Via onboarding, keys

13. 13

    Every tier emits metrics to observability, with per-tenant detail for the top tenants (or in logs and ClickHouse), since per-tenant metric labels explode cardinality, to spot lag, hot partitions and slow queries.

    Components Observability, Stream Processor, Event Stream · Via metrics

Sequence diagram

The same walkthrough as a UML sequence diagram, one lane per layer or group (20 components in all), left to right in the order a request reaches them, numbered by step. A loop on a lane is a step that stays inside that group. Solid arrows are synchronous calls, dashed are asynchronous.

Key components

Related walkthroughs

• Metrics and monitoring systemHow a monitoring system scrapes metrics from every service, alerts through grouped and routed notifications, and keeps years of data in object storage.SaaS & data
• Log aggregation pipelineHow a log pipeline collects lines from every host, buffers them in a topic, indexes only labels, and keeps compressed text in object storage.SaaS & data
• Time-series databaseHow a time-series database appends samples behind a write-ahead log, compresses them into immutable blocks, and answers range queries by label.SaaS & data