Multi-Agent Applied Builds: interview questions

Production questions of the kind asked in applied AI engineering interviews.

1. Larkspur's support desk is built like the hotel system: a coordinator, a compliance checker over account-privacy rules, and a billing agent. A customer asked for the billing email of another company's account and got it. The trace shows the compliance checker never ran. How do you find out why, and how do you stop it happening again?

What a strong answer covers:

• Reads the node path for that turn (stream_mode="updates"), then the coordinator's reply: the checker runs only when the reply names it, which the prompt asks for and nothing forces
• Makes the check structural: an unconditional edge from the guest message to the checker, instead of routing by mention
• Enforces privacy in the tool as well, the way the hotel's authorizer makes guest_name read as NULL, so a skipped check cannot leak the field
• Adds the request to the regression table and tracks its pass rate over several runs, since one green run proves little

2. The coordinator forwards to the billing agent whenever its reply contains billing_agent or "billing agent". A reply saying "I can't involve the billing agent until you confirm the refund date" forwarded anyway. A teammate proposes raising the confidence threshold from 50 to 80. What do you say?

What a strong answer covers:

• The threshold does not touch the cause, string matching on reply text; in the router lab, thresholds from 71 to 90 turn the known-issue test green by ending the turn early, not by routing correctly
• It also costs real traffic: the assessor scores a request with missing details at 50 to 79, so those customers get "please repeat" instead of a question about the missing detail
• The fix is a typed router: structured output with a Literal route, read by a pure function, as in notebook 04b
• Keeps the known-issue test, which flips to passing when the fix lands, and unit-tests the router on hand-built states with no model

3. Larkspur runs a research team for its support engineers: a planner, three parallel searchers, a writer and a quality checker with a two-draft cap. How do you know it is still working, and what it costs, a month after launch?

What a strong answer covers:

• Logs per run the number of search agents against planned queries, the number of drafts, and whether the run ended approved or stopped by the cap
• Watches the share of runs stopped by the cap: a rise means the writer or the sources got worse, since the cap hides quality problems from the user
• Tracks the share of findings whose URL was one the search returned (url_verified), the early sign of invented sources
• Reports cost and latency per run, which scale with queries times results and with drafts, and re-runs a fixed set of handbook questions after every prompt or model change

4. To speed up refunds, Larkspur splits billing into two parallel agents, one per region, both with a refund tool. A customer was refunded twice. What went wrong, and how do you design it?

What a strong answer covers:

• Parallel branches run the same request at the same time, so any write tool in both branches can fire twice
• Gives the branches read-only tools and makes the one write after a single choice, as the hotel's branches report availability but cannot book
• Adds an idempotency key to the refund tool so a retry or a duplicate call cannot pay twice
• Checks the wiring: one message key per branch, sync nodes and a join so the merge runs once, and a test that counts merge runs and rows written

5. You ship a deep agent to Larkspur's support engineers, built like the long-term memory agent of lesson 3 of T8.2 (now taught in T8.2). Someone sets USE_SANDBOX=true expecting cloud isolation, and the memory panel shows one engineer's notes in another's session. What is wrong?

What a strong answer covers:

• The script's comment promises a LangSmith sandbox, but the flag selects LocalShellBackend, which runs shell commands on the host with no isolation
• FilesystemBackend(virtual_mode=True) blocks path traversal; it is not a sandbox either, so code execution needs a real sandbox backend and approval on execute (interrupt_on)
• Long-term memory is keyed by CURRENT_USER_ID, fixed to default-user, so every engineer reads and writes one namespace; the key must come from the authenticated user
• recall_memories matches a substring, so it misses paraphrases; a store with search, and a review of what gets saved, come before scale