Deep Research Agent

Research process is: a question, hunt on the web and in your files, a cited brief. Failed hunt should look incomplete, not confident-fake.

G11 covers one slice: bounded, read-only research with citations. It cannot send email or buy anything.

End to end, as “Who is Acme’s CEO, with sources?”

1. You authenticate. Budgets and rate limits lock in. First signal in seconds.
2. A planner splits work: public web vs your Drive. Web never inherits Drive tokens.
3. Subagents search. Wikipedia timeout is “couldn’t check,” not “not found.”
4. Findings go to a ledger. A grader asks: enough coverage, or one more hop?
5. A synthesizer writes only from that ledger. Every fact needs a citation.
6. A verifier and sanitizer strip bad links/HTML. Gaps stay visible.

That’s it: plan → search in isolation → cite or show the hole → stop. External actions stay out.

Source: Sources and Full Design. Use the Deep Dive for state and security mechanics and the Cheat Sheet for rehearsal.

Build a read-only research agent for public web and a user's private documents: about 100,000 daily users and 40,000 research runs/day, p95 under 90 seconds, first signal under 3 seconds, 3–12 steps/run, cited claims, and about $0.20/run. A failed branch yields an explicitly incomplete answer, not fabricated certainty.

The easiest way to frame requirements in an interview is:

Functional = what the system does. Non-functional = how well it does it and what constraints it must satisfy.

Functional requirements — what the system must do

1. Authenticate and rate-limit.
2. Plan a bounded research task.
3. Search web and user documents with separate privileges.
4. Merge typed evidence.
5. Check coverage and replan if useful and budget remains.
6. Synthesize with citations.
7. Verify claims and sanitize rendered output.
8. Stream status and return explicit gaps.

Non-functional requirements — how well / under what constraints

Interview shortcut

If asked “What are the requirements?”, say:

“Functionally, plan bounded research, search public and private sources separately, cite every claim, and show gaps. Non-functionally, under 90 seconds, about 20 cents, no cross-user leak, and no invented certainty.”

The planner and synthesizer use a reasoning-tier LLM (large language model); narrower web/document agents can use smaller models. The supervisor is bounded by deterministic hop, budget and no-progress rules. Web agents never inherit private-document access.

Step-by-step architecture

1. Authenticate the user, apply run budgets and rate limits, and emit a first progress signal quickly.
2. A reasoning LLM makes a small dependency-aware plan; deterministic controls cap hops, elapsed time, calls and spend.
3. Independent web subagents search public sources in parallel while a separate document subagent retrieves only documents that this user may read.
4. Each branch returns typed findings and citations, not instructions. A reducer appends and deduplicates evidence in a user-scoped ledger.
5. A coverage grader compares findings with the plan. It may request a capped replan; repeated delegation without new evidence stops.
6. A synthesizer writes claims grounded in the ledger; a verifier checks claim-citation support and marks unavailable or uncovered areas.
7. An egress sanitizer removes image/markup fetches and disallowed links before streaming a cited answer or clearly labelled partial result.

Isolation is the hard boundary. Web pages and retrieved content are untrusted. The web agent gets no private-document credentials. Prompt text cannot enforce this. The final rendered answer is also an egress path, so strip Markdown images and fetch-inducing markup. Never share private-answer caches across users.

Parallel merge must be correct. If branches write a single summary field, a hand-rolled last-write-wins merge can silently drop findings (LangGraph raises InvalidUpdateError instead when the key has no reducer). Store a list of typed evidence with an append/concatenate reducer, provenance and dedupe. Sum budget consumption across branches.

Termination matters. Use a hard hop cap (the source gives eight as an example), per-run money/tool limits and no-progress detection. Distinguish no evidence found from a failed source. On a cap or outage, provide a partial answer with the missing section named.

Cost and latency: isolate subagent contexts; use smaller models for narrow extraction and a stronger model for planning/synthesis; cap rounds; deduplicate searches; cache public facts and user-scoped private work separately; measure cost by branch, hop and synthesis tokens. Parallelism helps only when tasks are independent and provider limits permit it.

What to score

Score outcome and trajectory, not only the final paragraph.

What to test

• Poisoned web pages
• Missing documents
• Conflicting sources
• Empty search (EMPTY)
• Unavailable tools (UNAVAILABLE)

Audit the evidence ledger and the link sanitizer, not only the final answer.

Related AWS research platform

This is my own build of the same product. It uses API limits, Bedrock guardrails, Redis/RDS memory, a search–summarise–write–critic loop, and observability.

Its source does not give per-user ACL (access control list) or deterministic release gating. Do not copy those defaults into this enterprise case.

When to use multiple agents

A supervisor–worker split is worth it for parallelism, distinct permissions, or context isolation (web vs Drive).

If the decomposition is predictable, a deterministic workflow is simpler.

“I would clarify citation quality, private-document permissions and the partial-answer contract first. The planner builds a bounded research plan, then public-web workers and a separate user-scoped document worker gather typed findings in parallel. Their evidence is appended into a user-scoped ledger; a coverage check can trigger a limited replan. A stronger LLM synthesizes the answer, and a claim verifier plus egress sanitizer ensures citations support the claims and rendered output cannot leak private content through images or links. I enforce hop, time and spend caps, return explicit gaps on failure, and measure both answer quality and the path the agents took.”

Read with the unchanged source. The Main guide gives the interview flow; the Cheat Sheet is the recall card.

The public-web subagents and private-document subagent have different tool credentials. A public branch must not receive user document scope, even if a web page tells it to request that scope. Retrieved text is evidence, never a system instruction. The private branch is scoped to one authenticated user and current access rights. Tenant/user identity belongs in retrieval and cache keys, not only in the prompt.

The final answer can itself leak data if it renders attacker-controlled Markdown images or external links with secret query parameters. Normalise typed findings, verify citations, and sanitize output at the egress boundary. Test injection in web pages and uploaded documents. A cross-user leak target of zero is more important than an average quality score.

A run records user/run ID, goal, dependency plan, typed evidence, tool errors, draft, coverage score, hop/replan counters and budget. Parallel branches should append to an evidence list with provenance and dedupe; a scalar summary with last-write-wins semantics loses a branch. Budget counters aggregate across branches. An evidence record should identify source URL or document ID, access scope, retrieved timestamp, quoted span/locator and claim relevance.

The planner may request 3–12 steps, but the supervisor enforces a hard cap and no-progress rule. If a branch repeats the same delegation without new evidence, stop. EMPTY means a successful search found nothing; UNAVAILABLE means a tool failed. The synthesizer must not treat both as proof that a claim is false.

40,000 runs/day is about 28/minute; a 3× peak is about 100/minute. With roughly 70 seconds active time, that suggests about 120 concurrent runs and around 170 slots with 70% utilisation. Validate against observed arrival bursts and service-time distribution; p95 time is not a formal mean for Little's Law. First signal under 3 seconds should be a useful status or partial evidence, not a fake answer.

Context isolation is a large cost lever: a web branch sees its task and relevant snippets, not the entire conversation or other branch histories. Use small models for narrow search summaries and a stronger reasoning model where planning/synthesis needs it. Prefix caching helps repeated stable instructions; public topic caches differ from user-scoped private caches. Track tokens into synthesis, cost per branch, searches per report and replans, then tune budgets without erasing necessary coverage.

The coverage grader asks whether evidence addresses each research subquestion and whether sources agree. Its score is not the release decision by itself. Evaluate citation correctness and claim support with human-labelled examples, including conflicting sources, stale pages, tool failures and adversarial instructions. Correct a misleading grader and then inspect its score distribution; a post-fix score drop can reflect better measurement rather than worse answers.

Trajectory metrics include hops, duplicate searches, misrouted private queries and budget exhaustion. Outcome metrics include task completion, grounded-claim rate, citation-free claims, explicit gaps, latency and cost. Track both, because a good-looking answer can have unsafe routing and a safe trajectory can still miss the user's question.

The AWS platform variant supplies useful concrete infrastructure: API key/Redis limits, Bedrock input/output guardrails, cache and vector memory, a search–summarise–write–critic loop, gateway tracing, Postgres/pgvector and deployment automation. Its lack of per-user ACL (access control list) and deterministic release gate remains a gap for the anchor's requirement. The supervisor-worker variant uses parallel subquestions and an append reducer; choose it when permissions or independent work justify agent boundaries. For predictable decomposition, a deterministic workflow avoids supervisor cost and loops.

Main · Deep Dive · Unchanged source

What counts as a valid citation? Which documents are private? What is the partial-answer rule? Is $0.20/run hard? What do failed tools mean?

Auth → bounded LLM plan → parallel public-web + user-scoped document agents → typed evidence ledger → coverage check / capped replan → LLM synthesis → claim verification → egress sanitizer → cited answer + gaps

Keep web and private-doc credentials separate. Append branch evidence; never last-write-wins a shared summary.

40K runs/day ≈ 28/min; 3× peak ≈ 84/min, rounded up to 100. Roughly 120 concurrent at a 70s mean run time, ~170 slots at 70% utilisation. p95 <90s, first signal <3s, about $0.20/run. Hard hop, tool, time and budget caps.

EMPTY ≠ UNAVAILABLE. If evidence is thin or a branch fails, state the gap. Strip image/markup fetches and unapproved links in the final answer. Private caches are user-scoped.

Check claim-citation support, coverage, cross-user leaks, p95, first signal, cost, hops, routing and no-progress loops. Test poisoned pages and missing tools.

60-second close: “I use multiple agents only where parallel research and permission isolation help. A bounded planner delegates, a typed ledger merges findings, and a verifier plus egress filter turns them into a cited answer with explicit uncertainty.”

Every extra search round makes the answer better grounded and the run slower and dearer, and the web pages that ground it are also the attack surface.

A research agent does not fail because it is hard to call a model. It fails when an unbounded fan-out quietly spends the budget. It also fails when a fetched page reaches data it should never have seen. This page covers the group's prompts in one read for the day before. It pairs one worked whiteboard design with one deployed system and one mock prompt.

Sections and tables marked (own construction) were built for this page from the sources' arguments and are not in the sources verbatim.

The first eight minutes buy every later answer. Twenty minutes on requirements leaves the design unfinished. Two minutes leaves it generic. Rehearse against a clock until the pacing is automatic.

Make three opening moves, every time. Restate the problem in one sentence. Ask for the eight numbers: throughput, latency, horizon, accuracy, cost, autonomy, data class and recovery. Then name the oracle and cut scope: "how will we know a run succeeded", and "I'll treat X as out of scope unless you want it in".

The prompt: design a read-only, citation-grounded research agent for ~100K daily users. It searches the public web and the user's own documents, then writes an answer in which every claim is cited.

Close requirements by restating the whole system as one sentence and asking "is that the system?". It costs 20 seconds. I'd expect an interviewer to mark it as a strong signal, because it is what a technical lead does at the start of a real project.

"A read-only research agent for about a hundred thousand daily users, forty thousand runs a day, under ninety seconds at p95 and twenty cents a run, where success means every claim is cited to a source a reviewer would accept. Is that the system?"

A requirement that cannot fail a test is a preference. "Well-researched" is a preference. "Every claim cited to a source a reviewer would accept" is an oracle, and only an oracle shapes the architecture.

The must-haves are five (the MoSCoW split is own construction; the items are the source's). Decompose a question into a dependency-aware plan. Search the public web and the user's documents in parallel. Collect findings into one deduplicated evidence ledger. Synthesise an answer with citations. Verify each claim against the evidence before streaming it with a coverage note.

The should-haves come after the core path is trusted. Probe a cache before planning. Replan when coverage is thin. Grade agreement between sources. Answer "not enough evidence" rather than guess.

Read-only is a requirement, not an omission. There are no writes, so no approval subsystem is needed. Say so explicitly, because the interviewer is listening for whether the autonomy question was asked.

Every must-have then needs an owner in the architecture (own construction).

State is the contract between the planner, the sub-agents and the synthesiser. Parallel branches that write one key under a last-write-wins rule lose all results but one. LangGraph refuses this with an InvalidUpdateError when the key has no reducer, but a hand-rolled merge loses the data silently. That is the fan-out data-loss defect.

The anchor's state carries seven things. Run and user identity. The goal. A dependency-aware plan. An evidence list with a concatenating merge rule. Tool errors. The draft and its verification score. Step and replan counters, and a budget dict summed across branches.

The mock (#103) names the fix in one line: any key more than one node writes needs a reducer that accumulates. The same mock warns against the scalar version of the bug. Several agents writing one summary key through a hand-rolled last-write-wins merge means the final answer reflects only the last one to run. Accumulate findings into a list, then synthesise once at the end.

The organising rule is privilege separation first, then cost. Web sub-agents read untrusted content and hold no user-document scope. Only typed findings cross back to the parent, and everything after the schema boundary is user-scoped.

The source's flow, redrawn as text:

Question + cache probe
        │
Plan (reasoning tier): decompose?
   ┌────┴─────────────────────┐
Web sub-agents             Document sub-agent
(untrusted, NO user        (user-scoped)
 doc access)                   │
   └────┬─────────────────────┘
Schema boundary — typed findings only
        │
Evidence ledger (user-scoped, dedups sources)
        │
Grade coverage + agreement
        │
Synthesize with citations
        │
Verify claims against evidence
        │
Streamed answer + citations + coverage note

The same system with its planes and trust zones (own construction): a control plane that changes by release, and a data plane split by who may see what.

 ╔══════════════════════ CONTROL PLANE (changes are releases) ══════════════════════╗
 ║ prompts per role (versioned) · tier routing · hop/step caps · budget ceiling      ║
 ║ egress allowlist for rendering · cache thresholds · grader rubric · eval gates    ║
 ╚══════════════════════════════════════╤════════════════════════════════════════════╝
                                        │ configures every box below
 ╔══════════════════════ DATA PLANE (calls are requests) ═══════════════════════════╗
 ║                                                                                   ║
 ║  ENTRY   user ─> authN + rate limit ─> input guardrail ─> CACHE PROBE ─ hit ─> out║
 ║                                                              │ miss               ║
 ║  PLAN                                     PLANNER (reasoning tier, plan + budget) ║
 ║                                             │ dependency-aware sub-questions      ║
 ║  ┌──────────── UNTRUSTED ZONE ────────────┐ │ ┌──────── USER-SCOPED ZONE ───────┐ ║
 ║  │ web sub-agents × N (small model)       │<┴>│ document sub-agent (user ACL)   │ ║
 ║  │ fetch · read · extract  NO doc scope   │   │ retrieve · extract              │ ║
 ║  └───────────────┬────────────────────────┘   └───────────────┬─────────────────┘ ║
 ║                  └──────── SCHEMA BOUNDARY: typed findings ────┘                   ║
 ║                                        │                                          ║
 ║  SYNTH   evidence ledger (dedup, user-scoped) ─> grade ─ thin ─> replan (capped)  ║
 ║                                        │ enough                                   ║
 ║          synthesise w/ citations (reasoning tier) ─> VERIFY claims vs evidence    ║
 ║                                        │                                          ║
 ║  OUT     EGRESS FIREWALL on render (strip images, non-allowlisted links) ─> stream║
 ║                                                                                   ║
 ║  OBSERVE trace per run, sub-agent traces linked · tokens + $ by branch · grades   ║
 ╚═══════════════════════════════════════════════════════════════════════════════════╝

The same flow for viewers that render Mermaid (own construction):

Read the components in dependency order (own construction; the components are the source's).

Sub-agent isolation is the single largest cost lever in this design. A parent that accumulates raw pages resends every token on every later step. A parent that receives one short conclusion per sub-agent grows by hundreds of tokens, not thousands.

Context rot compounds the cost problem. Model accuracy degrades as the window fills, and unevenly, so information stuck in the middle is retrieved less reliably. A run with three well-chosen observations often beats the same run stuffed with twelve.

Isolate when a subtask's intermediate steps are not needed downstream. That is the common case for research, verification and extraction. Share context only when steps must reason across each other's intermediates, which is rarer than it feels. On a ten-step run, isolation can shrink the parent's input tokens a lot, but sub-agents bill their own tokens, so measure the total.

Isolation gives up one thing: auditability across the boundary. Store each sub-agent's full trace and link it from the parent's span. Otherwise a wrong conclusion becomes unexplainable.

Isolation is also the security boundary. A web sub-agent that never holds user-document scope cannot leak a private document, whatever a fetched page tells it to do. Section 7 builds on this.

Numbers said out loud are the senior signal. Derive them in front of the interviewer rather than quoting them. The derivation is the part being scored.

Little's Law says work in flight equals arrival rate times mean time in system. It is the one formula that turns a throughput number into a capacity number. Name it by name.

Three levers bring the naive run under $0.20. Sub-agent isolation means the parent sees hundreds of tokens, not raw pages, so the planner's calls stay cheap — though each sub-agent still pays for the pages it reads. Prefix caching covers the stable instructions and tool definitions. Tier routing sends extraction and summarisation to a small model and keeps the reasoning tier for the plan and verification only.

At 10x, I'd expect provider quota to bind before compute. Name it first, then the arithmetic, then which lever to pull and what it costs.

The lethal trifecta is three capabilities that are dangerous together. They are reading private data, processing untrusted content, and communicating externally. An agent holding all three can be told by a web page to send a user's documents somewhere.

This agent reads private data and processes untrusted web content. It has no external communication tool, so the third leg looks absent. It is not. The rendered answer is itself a channel. A Markdown image whose URL encodes private text is exfiltration the moment the client fetches it.

So an egress firewall on the rendering path is mandatory. It strips Markdown images, non-allowlisted links and fetch-inducing markup. Web sub-agents hold zero user-document scope. An injection in a fetched page cannot reach the private corpus, even in principle. That is privilege separation, and it should be stated in those words.

Prompt defences are rate-reducers, not boundaries. Keep them, and never present them as the control. The boundary is the gateway, the scope separation, and egress control on both the tool path and the rendering path.

A research run can fail by stopping too early or by never stopping. Stopping early abandons solvable work. Never stopping burns budget on unsolvable work. Termination needs care in both directions.

The mock (#103) combines three defences, all read by one routing function. An explicit done flag from the supervisor. A hard hop cap. A no-progress check that compares this delegation with the last one.

MAX_HOPS = 8

def route(state) -> Literal["agent", "give_up", "end"]:
    if state["task_complete"]:
        return "end"
    if state["hops"] >= MAX_HOPS:
        return "give_up"                       # partial results, with a reason
    # Same agent, same request as last hop -> the loop is not converging
    signature = f"{state['next_agent']}::{state['messages'][-1].content[:200]}"
    if signature == state.get("last_delegation"):
        return "give_up"
    return "agent"

The no-progress signature is the piece people leave out. A hop cap alone still lets a supervisor and worker burn the whole budget rejecting and retrying the identical request. It only bounds how much they burn. The signature catches the loop on hop two instead of hop eight.

The canonical multi-agent failure is exactly this: the supervisor rejects, the worker retries the same way, the supervisor rejects again. The third defence is feedback specific enough for the worker to change its approach.

Every ceiling returns partial results with a reason rather than raising. Add the per-run budget as a fourth ceiling: the anchor's budget dict, summed across branches, stops the run at $0.20. The deployed platform (#12) uses the same shape. Its Critic loops back to Search while under agent_max_iterations, and a YES or an exhausted budget ends the graph.

A research answer is only as honest as its gaps. Ten parallel workers where one fails should produce nine findings and a gap note, not nothing. The wrong answer is losing nine successes to one failure because no one decided.

Rehearse "what breaks first?" most of all. "Hallucinations" is the wrong answer. The right shape names a symptom, a mechanism, a metric and a fix in one breath:

"The web search tool returns empty result sets during a partial outage. The agent treats empty as valid, and we ship an ungrounded response with no citations. The signal is citation-free answer rate. The fix is a typed result distinguishing empty from unavailable."

The ladder below applies that shape to each component (own construction).

A design that cannot be falsified cannot be improved. Name the one metric whose value would say the architecture was the wrong bet.

Here that metric is the grade distribution. If most runs grade poorly even after correction, the corpus is the constraint, not the architecture, and the wrong thing was built.

Evaluate at two levels. Outcome evaluation asks whether the final answer was right. Trajectory evaluation asks whether it took a sensible path: did the planner decompose sensibly, did agents stay in role, how many hops. Trajectory matters more in a multi-agent system. A run that reaches the right answer through six wasted hops is one prompt change away from not reaching it.

Debug a bad run from the trace keyed by run ID. Record per hop which agent ran, why it was chosen, what it saw and what it returned. The routing reasoning is the highest-value thing to log, because many bad multi-agent outputs come from how work was split and routed, not from one weak agent. Find the first hop where the trajectory diverged.

The Research Platform is my own build of the same product, running on AWS. Given a topic, it researches, writes a full report, safety-checks it, caches it and remembers it. Read it once as evidence that each whiteboard box is a real component.

It has nine layers, and one request passes through them top to bottom.

The delta from the anchor is the infrastructure. Four pieces carry it.

The gateway means an outage, rate limit or price change is a config change, not a code change in five places. The guardrails run on both sides, because a model can produce harmful output from a benign prompt. The input check runs on raw user input, not on an agent's interpretation of it. pgvector turns the cache from short-lived to durable. A report evicted from Redis is still found at ≥ 0.88, and a related one in the 0.50–0.88 band seeds the Writer. PyRIT attacks through the same auth, rate-limit and guardrail path as a real user. A passing run is therefore genuine evidence the defences hold, not a test of a mock.

Be precise about its shape. Layer 3 is a pipeline with a loop and a gate, closer to single-agent multi-step than to supervisor and workers. Its agents share one context and one tool set. What earns the split is the Critic's independent judgement and the bounded retry.

Name what it lacks, because that is the coverage-map habit. It has no per-user access control: one API key, no tenant model, so every user sees every cached report. That is fine for a research tool and wrong for an enterprise one, and it is exactly the anchor's per-user isolation requirement. It also has no deterministic release gate, since the judge scores are observability, not a gate.

The mock prompt is "design a research system with a supervisor and workers". It is the anchor's implementation view. The supervisor plans N queries, workers execute them independently in parallel, and a synthesiser merges.

State carries findings with operator.add, so parallel writes accumulate. Bound it with a hop cap and a per-run model-call budget. Then volunteer the honest trade-off. The supervisor's planning call is pure overhead if the query decomposition was predictable, and in that case hardcode it.

Parallel is right here because the queries are independent. Parallel costs N calls at once, merge logic and partial failure. It buys the latency of the slowest branch rather than the sum. Sequential is right only when a later step needs an earlier one's result.

Justify multi-agent by naming what it buys. The three honest reasons are real parallelism, distinct tool permissions per role, and context one agent cannot hold. This design has all three. It fans out searches, web agents hold different permissions from the document agent, and isolation keeps the parent's window small. If none of the three can be named, use one agent with more tools.

Four costs belong in the answer (from #103). A supervisor adds one model call per hop. Passing full history to every agent makes hop N pay for hops 1 to N-1, so pass a summary instead. Constrain routing with a Literal over the real agent names so the supervisor cannot route to one that does not exist. Parallel agents over one corpus duplicate retrieval, so share one retrieval step upstream.

The binding constraint in the round is time. Miss the first 8 minutes and every later answer is guesswork. Miss the last 5 and the levelling signal is gone.

Cover the whole design at consistent depth first, then offer depth explicitly: "I can go deeper on the evaluation layer or the cost model — which is more useful to you?" Candidates who go deep unprompted usually pick the component they know best. They then run out of time before the failure and measurement sections, where the levelling signal lives.

The follow-up bank, with the shape of a strong answer:

Recover a round that goes sideways out loud. Behind on time: say so and reprioritise aloud. The interviewer keeps redirecting: stop defending the path, follow, and ask what they want to assess. A wrong number: correct it immediately. Self-correction is a strong signal, and an uncorrected error that was noticed is a weak one. An unfamiliar domain: say so and ask two questions that make it reasonable. The failure is not running out of time. It is running out of time without saying so.

The two-minute spoken answer (own construction from the source's design):

This is a read-only research agent, forty thousand runs a day, peaking at three times the mean, under ninety seconds at p95 with a first signal in three, and twenty cents a run. Success is every claim cited to a source a reviewer would accept. A planner on the reasoning tier decides whether to decompose and writes a dependency-aware plan. Web sub-agents fetch and read in their own windows and hold no access to the user's documents. A separate user-scoped sub-agent reads those. Only typed findings cross the schema boundary into a deduplicated evidence ledger with a concatenating merge, so no parallel branch is lost. A grader checks coverage and agreement, the synthesiser writes with citations, and a verifier checks every claim before the answer streams with a coverage note. At peak that is about a hundred runs a minute and around a hundred and twenty in flight, so I would provision around a hundred and seventy. A naive seven-step run blows the budget, and sub-agent isolation, prefix caching and tier routing bring it back. On safety, the agent reads private data and untrusted pages. The rendered answer is its outbound channel, so rendering goes through an egress firewall. The metric that would prove me wrong is the grade distribution.

The lines that carry the round (own construction from the source's arguments):

1. "Read-only, so no approval subsystem. I'm saying that on purpose."
2. "Sub-agent isolation is the single largest cost lever. The parent sees hundreds of tokens, not raw pages."
3. "Evidence merges by concatenation. Last-write-wins silently loses branches."
4. "A hundred a minute, a hundred and twenty in flight, a hundred and seventy provisioned."
5. "The rendered answer is a channel. Egress control goes on the rendering path too."
6. "Web agents hold zero document scope, so an injection can't reach the private corpus even in principle."
7. "Empty is not unavailable. Type the difference."
8. "If most runs grade poorly after correction, the corpus is the constraint, not the architecture."

The interviewer's pivot after a good design is "each report costs too much." Cost here is search fan-out times synthesis context. The answer is the self-drill card.

Each lever maps to a part of the design already drawn. The round cap is the hop cap and budget dict from section 8. Deduplication is the evidence ledger from section 4. Compression before synthesis is sub-agent isolation from section 5. The topic cache is the deployed platform's smart lookup from section 11, with its 0.85 and 0.88 thresholds.

The multi-agent mock adds the accounting step that must come first. Split cost per run by agent and by hop, so it is clear whether routing calls, agent calls or retries dominate. Collapse agents whose only job is passing messages along. Route only hard steps to the expensive model. Ship every cut with an eval number showing quality held.

Every strong cost answer follows four verbs in order. Measure, by tracing tokens and dollars by branch first. Route, with the small model for extraction and the reasoning tier for plan and verification. Bound, with caps on rounds, hops and the per-run budget. Cache safely, never serving a private-document answer across users.

The levelling rubric describes what to say, not who the candidate is.

• The first eight minutes set the eight numbers, the oracle and the scope cut, and end with "is that the system?".
• Requirements are testable: ~40K runs/day, p95 under 90s, first signal under 3s, $0.20/run, read-only, per-user isolation.
• State carries a concatenating evidence list and a budget summed across branches, so no parallel result is lost.
• The architecture separates an untrusted web zone from a user-scoped zone, joined only by typed findings.
• Sub-agent isolation is the largest cost lever and also the security boundary.
• The arithmetic is 100 runs/min at peak, ~120 in flight by Little's Law, around 170 provisioned.
• The rendered answer is an exfiltration channel, so an egress firewall on rendering is mandatory.
• Termination uses a done flag, a hop cap, a no-progress check and a per-run budget, each returning partial results.
• Every failed branch degrades to a gap note, and empty is typed apart from unavailable.
• The grade distribution is the falsifying metric, and trajectory evaluation matters more than outcome alone.
• The deployed platform proves the boxes are real, and lacks per-user access control and a release gate.
• The supervisor–worker build fans out N independent queries, with the honest caveat that predictable decomposition should be hardcoded.
• The 45-minute script, the follow-up bank and the rubric decide the level more than the diagram does.
• The cost pivot is answered by capping rounds, deduplicating, compressing and caching by topic.

1. What are the eight numbers, and which one makes an approval subsystem unnecessary here? Throughput, latency, horizon, accuracy, cost, autonomy, data class, recovery. Autonomy: the agent is read-only.
2. Why must the evidence list use a concatenating merge? Parallel branches writing one key under last-write-wins lose all results but one. LangGraph raises InvalidUpdateError when the key has no reducer; a hand-rolled merge loses them silently.
3. Derive the concurrency number. 40K/day peaked 3x ≈ 100 runs/min; at ~70 s mean time in system, Little's Law gives ~120 in flight; at a 70% utilisation target, provision around 170.
4. What is the single largest cost lever, and what does it give up? Sub-agent isolation; it gives up auditability across the boundary unless sub-agent traces are stored and linked.
5. The agent has no email or webhook tool. Why is the trifecta still live? The rendered answer is a channel: a Markdown image URL can carry private text out when the client fetches it. Filter rendering.
6. A hop cap is in place. Why add a no-progress check? A cap only bounds the burn; the signature check catches an identical rejected retry on hop two instead of hop eight.
7. What is the strong answer to "what breaks first?" Search returns empty during an outage, empty is treated as valid, an uncited answer ships; watch citation-free answer rate; type empty apart from unavailable.
8. Which metric would prove the design wrong? The grade distribution: if most runs grade poorly even after correction, the corpus is the constraint.
9. Name two properties the deployed Research Platform lacks. Per-user access control, and a deterministic release gate.
10. When is the supervisor's planning call pure overhead? When the query decomposition is predictable; hardcode it instead.
11. What is the sixty-second cost answer? Cost is search fan-out times synthesis context: bound the rounds, dedupe, compress, and never research the same topic twice.

Fact-check sources (checked 27 Sep 2026)

• LangGraph docs: INVALID_CONCURRENT_GRAPH_UPDATE — parallel writes to a key with no reducer raise an error; silent loss needs a hand-rolled merge
• Liu et al. 2023, Lost in the Middle — information in the middle of a long context is used less reliably
• Chroma Research 2025, Context Rot — accuracy degrades as input length grows
• Anthropic Engineering: How we built our multi-agent research system — multi-agent runs use many more total tokens; isolation does not cut the bill by itself
• Little 1961, A Proof for the Queuing Formula L = λW — Little's Law uses mean time in system
• Claude docs: Models overview — current model tiers
• Claude docs: Pricing — per-token prices behind the $0.20 run budget
• Claude docs: Rate limits — provider quotas that can bind at 10x
• Simon Willison: The lethal trifecta for AI agents (16 June 2025) — private data plus untrusted content plus an outbound channel
• MSRC: How Microsoft defends against indirect prompt injection attacks (Jul 2025) — Markdown image exfiltration; prompt defences reduce rates but are not boundaries
• OWASP GenAI LLM01:2025 Prompt Injection — prompt injection has no complete prompt-level fix
• Cemri et al., Why Do Multi-Agent LLM Systems Fail? (arXiv 2503.13657) — failures spread across design, coordination and verification, not routing alone
• AWS Bedrock User Guide: Use the ApplyGuardrail API — guardrails can screen input before any model call
• tensorzero/tensorzero on GitHub — repository archived on 12 June 2026
• GroqDocs: Model deprecations — llama-3.1-8b-instant retired for free and developer tiers in Aug 2026
• OpenAI API docs: Deprecations — GPT-4o still offered
• OpenAI API docs: GPT-4o model page — GPT-4o still listed
• microsoft/PyRIT — Crescendo, Skeleton Key and XPIA (cross-prompt injection attack) attacks
• Reference build: app/auth.py and app/cache.py — one API key, no tenant model, shared report cache