Worked Example · 13 min — Start here. One travel-agent problem taken through every part of the framework.
Before this: The Twelve-Part Design Framework and Design Principles as Tie-Breakers
Why this matters
Principles stay abstract until you watch them repair a real diagram, so that is what this document does. We take a sample design for a travel assistant built from AI agents, list five structural faults a careful review found, and redraw it. Keep the habit of spotting those faults; the fixed diagram matters less. Most first-pass multi-agent sketches repeat the same five, including the ones you will draw under time pressure.
The brief: an AI travel assistant serving customers in three regions (AMER, EMEA, APJ), where one specialist agent each handles policy, pricing, search, booking and recommendation.
The original, and what was wrong with it
The first draft was a rigid pipeline. Five agents fanned out, and then all of their work squeezed through a single shared cache, a single retrieval stage, a single LLM (large language model) step and a single tool-calling stage, one after another. It needed five fixes.
1 · Rate limiting belongs inside the API gateway
In the draft, Rate Limiter hung off the load balancer as its own box, side by side with the gateway. No line left it, so anything sent there went nowhere. Rate limiting is not a route. It is a rule the gateway applies to every request. A separate box implies a component, and rate limiting is a behaviour.
2 · The semantic cache is checked before any agent spawns
The draft put the cache behind the five agents, so it was only consulted after they had all run. If a cache exists to cut LLM latency and cost, it must come before the costly work. Now the supervisor looks in the cache before doing anything else. On a hit, the answer goes back and no agent is spawned at all. In 12-part framework terms this is short-circuiting, and the placement decides whether the cache saves money or nothing.
3 · Retrieval is owned by each agent, not a shared stage
A single compulsory retrieval box after all five agents made every agent query the vector DB, whether it needed to or not. Search and Recommendation do draw on regional travel content. Booking and Policy rarely do. In the fix, each agent calls retrieval itself, within its own loop, and only when its task calls for it.
4 · Tool calling, MCP and external APIs sit alongside the agents
In the draft, all five agents passed in sequence through a shared LLM box, a shared tool-calling box and a shared MCP (Model Context Protocol) registry. That reads as though the request gets a single turn in total. Booking actually calls the Flight and Payment MCP servers during its own work; it cannot wait until four other agents finish. The redraw makes the LLM, the MCP registry, the external APIs and the vector DBs a shared backend that each agent calls on demand.
5 · A synthesiser agent replaces the missing aggregation step
Nothing in the draft explained how five parallel results became a single reply. The flow simply carried on as one line, as though merging took care of itself. It does not. In the fix, a named Synthesiser Agent gathers the output of each agent that ran and settles any clash between them before the user sees anything. A Policy ruling that overrides a Booking choice is the classic example.
The corrected architecture
On a cache hit, the user gets a reply and no agent starts. If it misses, the supervisor spawns just the agents this query calls for, and each one pulls from the shared backend only what its own task needs. The synthesiser merges their results into one answer, which is also saved to the cache.
Components
| Component | Purpose |
|---|---|
| Client | Web and mobile app for searching, booking and managing trips |
| Authentication | Runs first: SSO through OpenID Connect (an identity layer built on OAuth 2.0), issuing JWTs as tokens |
| Load balancer | Shares traffic across API instances |
| API gateway | A single policy layer that routes, validates and throttles requests |
| Supervisor agent | Reads the query and looks in the cache first; picks specialists to start only after a miss |
| Semantic cache | Checked straight away; on a hit, no agent is spawned |
| Specialised agents | Each pulls the domain context and tools its task needs; Booking reaches the payment gateway directly |
| Regional vector DBs | Shared retrieval that an agent queries when it chooses, not a stage on every request's path |
| MCP tool registry | Finds tools at run time; any agent that needs it can call it |
| External APIs | Flights, hotels, maps and payment, reached through MCP servers |
| Synthesiser agent | Combines outputs, settles conflicts and returns one coherent reply |
| Monitoring & logging | Tracks latency, tokens, failures and a trace for each agent |
Data flow, in order
- A user sends a request.
- Authentication confirms who they are, the load balancer picks an instance, and the gateway enforces rate limits before passing the request to the supervisor.
- First, the supervisor looks up the semantic cache.
- A hit returns at once, and no agent is spawned.
- On a miss, the supervisor starts just the agents this query calls for.
- Inside its own loop, an agent that needs context makes a tool call to the matching regional vector DB.
- In that same loop, it calls external APIs via the MCP registry.
- The synthesiser gathers the outputs, settles conflicts and writes one grounded reply.
- The user receives the reply, and a copy goes into the cache.
- Across the whole request, monitoring captures latency, token counts and a trace per agent.
Non-functional requirements
| Requirement | Solution |
|---|---|
| Scalability | Scale out horizontally, autoscale, balance load |
| Reliability | Deploy in several regions; retry; fail over |
| Availability | Regions run active-active |
| Performance | Semantic cache placed ahead of the fan-out; Redis; CDN; tuned vector search |
| Security | OAuth, JWT, encryption, RBAC, managed secrets |
| Cost | A cache-first design where the cache really does gate spawning; small models for easy tasks; routing by region |
| Observability | Metrics, logs, a distributed trace per agent, dashboards, alerts |
| Maintainability | Modular agents that each own their tools; CI/CD; infrastructure as code |
Trade-offs worth naming out loud
Shared retrieval service vs per-agent RAG (retrieval-augmented generation). One shared retrieval layer gives you a single place to enforce data residency and guardrails, but every agent then depends on that one service. Per-agent retrieval matches how frameworks such as LangGraph and CrewAI wire tools to each agent, yet without governance you end up running several copies of the embedding stack. You can defend either choice. What is actually wrong is placing RAG beside the agents as if it were one of them; it is infrastructure, or a tool.
Single agent vs multi-agent. Every extra agent adds coordination and a synthesis step you must pay for. Pay it when sub-tasks differ in their tools or guardrails, or must scale on their own. Booking and Policy plainly do. Module 07 sets out the full rule for deciding.
Semantic cache vs traditional cache. A traditional cache only hits on an exact repeat. A semantic cache also matches near-duplicates, so it can skip the entire fan-out rather than save a single call downstream.
Global vs regional vector DBs. Stores per region make data-residency rules easier to meet and bring latency down. One global store is simpler to run and keeps regions consistent with each other.
A mental model: the international airport
| Airport | System |
|---|---|
| Travellers | Users |
| Terminal entrance | Authentication |
| Security screening and bag scan in a single lane, not two | Rate limiting as part of the API gateway |
| Help desk that checks "did someone already answer this?" | Semantic cache, consulted before staff are dispatched |
| Control tower | Supervisor agent |
| Airline crews, each with its own manuals and radios | Specialised agents, each with its own retrieval and tools |
| Reference binders kept at each desk | Vector databases, consulted by any desk that needs one |
| Ground handlers, caterers and fuel trucks | External APIs |
| Dispatcher who writes one departure report | Synthesiser agent |
| CCTV and the operations room | Monitoring and observability |
Interview lens
Each of the five fixes turns into a question you can put to any multi-agent diagram, your own included:
- Is each box a real component, or a policy that should live inside another one?
- Does the cache come ahead of the costly work it is meant to avoid?
- Are agents made to retrieve when they have no need to?
- Does every agent loop and call tools by itself, or must the whole request squeeze through a single shared stage?
- Where do the parallel results meet, and who settles disagreements between them?
Checkpoint
- List the five structural fixes. Which principle from the last document does each one apply?
- Why does the position of a semantic cache decide whether it saves anything?
- When teams argue shared versus per-agent RAG, what mistake actually matters?
- Which job of the synthesiser is hidden when a diagram just "carries on as one line"?
- Redraw the corrected architecture without looking.
Key Takeaways
- The first draft was a rigid pipeline in which five agents squeezed through one shared cache, one retrieval stage, one LLM step and one tool-calling stage, and it needed five structural fixes.
- Rate limiting is a rule the API gateway applies to every request, not a separate box off the load balancer that leads nowhere.
- The supervisor looks in the semantic cache before doing anything else, and on a hit the answer goes back with no agent spawned.
- The cache's placement decides whether it saves money or nothing, because a cache consulted after all five agents have run avoids no work.
- A semantic cache also matches near-duplicate queries, so it can skip the entire fan-out rather than save a single call downstream.
- On a miss, the supervisor spawns only the specialists this query calls for, not all five by default.
- Each agent calls retrieval inside its own loop, because Search and Recommendation need regional travel content while Booking and Policy rarely do.
- The LLM, the MCP registry, the external APIs and the vector DBs form a shared backend that each agent calls on demand, so Booking reaches the Flight and Payment servers during its own work.
- A named synthesiser gathers the output of every agent that ran and settles clashes, such as a Policy ruling that overrides a Booking choice, before the user sees anything.
- The synthesised reply is written back to the cache so the next similar query can reuse it.
- Shared retrieval puts residency and guardrails in one place while per-agent retrieval risks duplicate embedding stacks, and the real mistake is drawing RAG as a peer of the agents.
- Extra agents add coordination and synthesis costs, which pay off only when sub-tasks differ in tools, guardrails or scaling, as Booking and Policy plainly do.
- Regional stores for AMER, EMEA and APJ ease data residency and cut latency, while one global store is simpler to run and keeps regions consistent.
- Monitoring spans every stage with latency, token usage, cache hit rate and a trace per agent.
Level calibration
On the travel assistant, the levels split on whether the candidate only fixes the diagram's flow or also owns how the fixed system runs across regions and teams.
| Level | Sounds like | Missing |
|---|---|---|
| Senior | "Rate limiting lives inside the gateway. The supervisor checks the semantic cache before it spawns anything, and each agent calls retrieval and MCP tools inside its own loop. A synthesiser merges whatever ran and lets Policy override Booking." | Which queries may be served from the cache at all, and what the synthesiser does when one spawned agent fails or times out. |
| Staff | "Shared retrieval versus per-agent RAG is a deliberate choice: shared for residency and guardrails, or per-agent with governance so embedding infrastructure isn't duplicated. I'd run it like a platform, with per-agent traces, cache hit rate and token usage on dashboards, smaller models for simple agents, and retries and failover per region." | Who owns each regional vector DB and the MCP tool registry, and the rule for when a new agent earns its coordination overhead. |
| Principal | "AMER, EMEA and APJ stores buy residency and latency, and one global store buys simpler operations, so I'd make that call region by region on an active-active footprint. Then I'd turn the five fixes into a review standard every team's multi-agent diagram has to pass." | Each region's residency rules and the real traffic mix, which only the customer can supply. Until then, [cache hit-rate target] and [p95 target] stay open. |
Next → The 60-Minute Whiteboard Method
Sources (checked 27 Sep 2026)
- OpenID Connect Core 1.0 — OIDC (OpenID Connect) is the identity layer on OAuth 2.0; OAuth alone does not authenticate a user
- MCP specification — Tools — clients discover tools with
tools/listand getlist_changednotifications, which is the dynamic discovery the registry relies on - Agents — CrewAI docs — each agent carries its own
toolslist, so per-agent tool wiring is the framework default - Semantic caching with Redis LangCache — semantic caches match similar questions by embedding similarity, not exact text
- GDPR Article 46 — transfers subject to appropriate safeguards — GDPR (General Data Protection Regulation) allows safeguarded transfers, so regional stores help with residency requirements rather than being what compliance means
12-Part Framework · 9 min — The general framework the worked example applies, to reuse on any design question.
Reference design — the corrected version
System design is the plan for how software parts work together to solve a business problem quickly, reliably and at scale. This page applies that plan to one sample system: an agentic AI travel assistant for customers in AMER, EMEA and APJ. It starts from a first-draft reference architecture and folds in five structural fixes that came out of two later reviews.
What this is: A first-draft architecture for a travel-agent system laid it out as a rigid pipeline. Five agents fanned out, then all passed, in order, through one shared cache, one retrieval stage, one LLM (large language model) step and one tool-calling stage. An architecture review found four points where that order does not match how a multi-agent system really runs. A separate blueprint, written independently, reached the same four fixes and added a fifth: a named Synthesizer Agent to fill the missing aggregation step. This page combines all five into one final architecture.
What changed, and why
The first draft needed five structural fixes. Four came from the architecture review and one from the earlier blueprint.
1 · Rate limiting moved inside the API Gateway
In the first draft, Rate Limiter was a separate box off the load balancer, next to the Gateway, with nothing flowing out of it. It was a dead end. Rate limiting is not a path a request takes. It is a rule the Gateway enforces on every request.
2 · Semantic Cache checked before any agent spawns
The cache used to come after all five agents had finished. A cache that is meant to cut LLM latency and cost can only do so if it sits ahead of the costly work. It is now the first thing the Supervisor checks, and on a hit the answer returns with no agent spawned.
3 · RAG owned by each agent, not a shared stage
A single required retrieval box after all five agents made every agent hit the vector DB, even agents with no use for it. Search and Recommendation do need regional travel content. Booking and Policy seldom do. So each agent now calls retrieval for itself, when it needs to.
4 · Tool calling, MCP, and external APIs sit alongside the agents
The draft pushed every agent through one shared LLM step, then a single Tool Calling step, then a single MCP (Model Context Protocol) Registry, as though the request had only one turn in total. The Booking Agent needs the Payment and Flight MCP servers while it is running, not after four unrelated agents have wrapped up.
5 · A Synthesizer Agent replaces the missing aggregation step — added by the prior blueprint
The draft never said how five parallel outputs turn into one reply. It just kept going as a single line, as if merging happened by itself. Now a named Synthesizer Agent combines the output of every agent that ran and settles conflicts between them before the user sees anything. A Policy decision that overrides a Booking choice is a typical case.
Architecture
The components and the regional footprint match the first draft. What changes is the structure: a cache hit now really is cheap, each agent uses its tools within its own loop, and all results pass through one accountable step before the reply leaves.
- 🟢 the request path
- 🟢 agents, each running its own loop in parallel
- 🟣 shared backend, called only when needed (dashed lines)
- 🟠 added: Synthesizer Agent
A cache hit (teal) returns before the Supervisor spawns any agent. On a miss, the Supervisor spawns only the agents this query needs. Each of them calls the shared backend on its own (the Foundation LLM, the regional vector DBs, the MCP registry and the external APIs) for just what its task requires. The Synthesizer Agent then combines whatever ran into a single reply, and that reply is written back to the cache so the next similar query can reuse it.
Key components
The list of components matches the first draft. The Gateway, the Cache and the agent-owned services have a new scope, and the Synthesizer is new.
| Component | Purpose |
|---|---|
| Client | The web and mobile app where users search, book and manage trips. |
| Authentication | Confirms identity through SSO with OpenID Connect (an identity layer on top of OAuth 2.0), using JWTs as tokens, before any other step runs. |
| Load Balancer | Shares traffic across API instances to keep the service available. |
API Gateway now holds rate limiting |
The one way in. Routing, request validation and throttling form a single policy layer rather than an extra hop. |
Supervisor Agent refocused |
Reads the query and checks the cache. Only when the cache misses does it choose which specialised agents to spawn. |
Semantic Cache now checked first |
Consulted as soon as the Supervisor has the query. On a hit, the answer goes back with no agent spawned. |
Search / Booking / Pricing / Recommendation / Policy Agents own their RAG and tools |
Each agent fetches its own domain context and calls its own tools. The Booking Agent talks to the Payment gateway itself, and nothing passes through a single shared LLM step. |
| Regional Vector DBs (AMER / EMEA / APJ) | Still the shared retrieval layer, but now something agents call into, not a stage that every request must pass through in turn. |
| MCP Tool Registry | Finds tools at run time. Any agent can reach it on its own, whenever it needs to. |
| External APIs | Flight, hotel, map and payment providers, called through MCP servers. |
Synthesizer Agent added |
Combines the output of every agent that ran, settles conflicts between them (for example, Policy overriding a Booking choice) and returns one coherent reply. |
| Monitoring & Logging | Records latency, token usage, failures and per-agent traces for the whole request. |
Data flow
The ten steps from the first draft, put in a new order so that cache-first really does come first.
- A user sends a request from the web or mobile client.
- Authentication confirms the user's identity. The Load Balancer sends the request to a free API instance. The API Gateway enforces rate limits and hands the request to the Supervisor.
- Before making any decision about agents, the Supervisor checks the Semantic Cache.
- If the cache hits, the stored answer goes straight back and no agent is spawned at all.
- If it misses, the Supervisor spawns the specialised agents this query actually calls for (Search, Booking, Pricing, Recommendation, Policy).
- Each agent pulls the context it needs from the right regional Vector DB, using a tool call within its own loop.
- Each agent calls the external APIs it needs (flights, hotels, maps, payment) through the MCP Tool Registry, again within its own loop.
- The Synthesizer Agent gathers every agent's output, settles conflicts and writes one grounded reply.
- The user gets the final reply, and the answer is stored in the Semantic Cache so later queries can reuse it.
- Monitoring and logging record latency, token usage and per-agent traces for the whole request.
Non-functional requirements
These are the same as in the first draft. The new order above simply makes the cache-first row true in practice.
| Requirement | Example solution |
|---|---|
| Scalability | Scale out horizontally, autoscale and balance load. |
| Reliability | Run in several regions, retry and fail over. |
| Availability | Regions run in active-active mode. |
| Performance | A semantic cache ahead of the fan-out, Redis, a CDN and tuned vector search. |
| Security | OAuth, JWT, encryption, RBAC and managed secrets. |
| Cost optimisation | A cache-first design where the cache really decides whether agents spawn, small models for easy tasks, and routing by region. |
| Observability | Metrics, logs, distributed tracing for each agent, dashboards and alerts. |
| Maintainability | Microservices, modular agents that each own their tools, CI/CD and infrastructure as code. |
Trade-offs worth naming out loud
The contrasts an interviewer wants to hear you draw.
Shared retrieval service vs. per-agent RAG (retrieval-augmented generation) — A shared Vector DB layer puts data-residency and guardrail enforcement in one place, but it ties every agent to that single service. Retrieval per agent mirrors how LangGraph- and CrewAI-style frameworks attach tools, yet without governance it duplicates the embedding infrastructure. Both positions hold up. The real mistake is drawing RAG as a peer of the agents, when it is infrastructure or a tool.
Single agent vs. multi-agent — Several agents bring coordination and synthesis costs. Those costs pay off when sub-tasks need their own tools, their own guardrails or their own scaling, which is plainly true of Booking and Policy here.
Semantic cache vs. traditional cache — A semantic cache hits on near-duplicate queries as well as exact repeats. That is why it can skip the entire agent fan-out instead of saving a single call further down.
Global vs. regional vector DBs — Stores per region (AMER/EMEA/APJ) make data-residency rules easier to meet and cut latency. A single global store is simpler to operate and keeps regions consistent with each other.
Mental model: an international airport
The airport comparison from the first draft, stretched to cover the cache gate and the Synthesizer.
| Airport | System |
|---|---|
| Travellers | Users |
| Terminal entrance | Authentication |
Security screening and bag scan a single lane |
API Gateway with rate limiting inside it |
| Help desk asking "have we answered this already?" | Semantic Cache, checked before any staff member is sent out |
| Control tower | Supervisor Agent |
| Airline crews, each with its own manuals and radios | Specialised agents, each with its own retrieval and tool access |
| Reference binders kept at each desk | Vector databases, used by whichever desk needs them |
| Ground handlers, caterers and fuel trucks | External APIs (flights, hotels, maps, payments) |
Dispatcher who writes one departure report from every crew added |
Synthesizer Agent |
| CCTV and the operations room | Monitoring, logging and observability |