AI applications, an interactive architecture walkthrough
AI code review bot: how the architecture works
How an AI code review bot queues pull request webhooks, gathers diff context in a sandbox, verifies model findings, and posts inline comments.
This is a learning explainer based on public engineering writing and general design practice. It is simplified, it is not the actual architecture of any company, and it may be inaccurate.
Interactive diagram
Overview
An AI code review bot reads each pull request and posts specific, line-level review comments. The code host sends a webhook for every new or updated pull request; the bot verifies it, acknowledges it immediately and queues a review job, because a review takes far longer than a webhook can wait.
A worker gets a short-lived token scoped to that installation, checks out the head commit in a sandbox without secrets, collects the changed hunks with surrounding context and related files, and runs linters and type checks. A language model then proposes findings with file, line and reason, a verification pass removes weak ones, and the rest are posted as inline comments and a check run. Developer reactions are tracked to measure precision. This is a simplified learning design, not any specific product's implementation.
Trade-offs and what changes at scale
More context finds more real problems but costs tokens and latency, so context is retrieved selectively. Posting fewer, better comments keeps developers reading them; precision is the metric that matters. Pull request content is untrusted input that can try to steer the model, so the bot runs with minimal permissions and never executes PR code with secrets. At larger scale reviews are cancelled when newer commits arrive, results are cached per commit, and model usage is budgeted per organisation.
How it works, step by step
The walkthrough above, as text. Each step lights up the components named under it.
When a developer opens or updates a pull request, the Code Host sends a webhook, and the Webhook Receiver verifies its signature and drops repeated delivery ids. It answers within a few seconds, because slow webhook responses are treated as failures.
The receiver only enqueues a job on the Review Queue; all real work happens later. A review takes minutes, far longer than a webhook can wait.
The Review Orchestrator takes the job, cancels any older review for the same PR, and gets a short-lived, read-mostly token for that one installation from the Token Service. The bot holds no long-lived credentials to any repository.
The Sandboxed Checkout clones the repository at the PR's head commit in an isolated environment with no secrets. Code from a fork must never run where it could read credentials.
The Diff Analyzer extracts the changed hunks and pulls in surrounding functions and related files from the Repo Context Index. A model that sees only the diff misses how changed code is called elsewhere.
Static Analyzers run linters, type checks and a secret scan on the changed files. Deterministic tools catch whole classes of issues more reliably and cheaply than a model.
The orchestrator sends the diff, context, tool results and the repository's review guidelines through the LLM Gateway, asking for concrete findings with file, line and reason. Asking for structured findings makes the next step possible.
The Finding Verifier re-examines each candidate finding against the code and drops vague or unsupported ones. A reviewer that posts many wrong comments is quickly ignored, so precision matters more than count.
The Comment Publisher posts verified findings as inline comments on changed lines and a summary check run with a status. Comments outside the diff are folded into the summary, since review comments can only attach to changed lines.
Each review and finding is stored in the Review DB with the commit it applied to. On the next push, findings already posted are not repeated.
Developer reactions, resolved threads and dismissals flow as Feedback Events into Quality Analytics, where precision is measured per rule and repository. Rules that are mostly dismissed are tuned or turned off.
Sequence diagram
The same walkthrough as a UML sequence diagram, one lane per layer or group (18 components in all), left to right in the order a request reaches them, numbered by step. A loop on a lane is a step that stays inside that group. Solid arrows are synchronous calls, dashed are asynchronous.
Key components
Intakedomain
- Delivery Dedup: seen delivery ids (redis)
- Review Queue: review jobs per pull request (kafka)
- Webhook Receiver: verify signature, ack fast
Source Controllayer
- Code Host: repos, pull requests, webhooks, checks API
- Developer: opens and updates pull requests
Review Pipelinedomain
- Diff Analyzer: changed hunks plus surrounding code
- Finding Verifier: re-checks each finding, drops weak ones
- LLM Gateway: model calls, limits, usage
- Model Provider: language model
- Repo Context Index: embedded symbols and files (pgvector)
- Review Orchestrator: one review per PR head commit
- Sandboxed Checkout: clone at the head commit, no secrets
- Static Analyzers: linters, type checks, secret scan
- Token Service: short-lived per-installation tokens
Outputdomain
- Comment Publisher: inline comments on changed lines, check run
- Feedback Events: resolved, dismissed, thumbs (kafka)
- Quality Analytics: precision by rule and repo (clickhouse)
- Review DB: reviews, findings, reactions (postgresql)
Sources
Public writing this explainer draws on. Read these for what the real systems actually do.
Make it yours
Copy this diagram and its walkthrough into your own workspace, then change anything: swap Postgres for DynamoDB, add a cache, rewrite the narration. Free, and it uses no AI credits.
Related walkthroughs
- AI agent platform with toolsHow an AI agent platform runs a model-and-tools loop on workers, checkpoints each step, scopes tools to the user, and pauses risky actions for approval.
- LLM gateway and routerHow an LLM gateway gives every team one API, enforces token limits and budgets, routes to the right model, and falls back when a provider fails.
- RAG chatbot over your docsHow a retrieval-augmented chatbot ingests company documents, retrieves permitted passages with hybrid search and reranking, and answers with citations.
This is a learning explainer based on public engineering writing and general design practice. It is simplified, it is not the actual architecture of any company, and it may be inaccurate. Real systems differ in the details. Browse the library.