ReAct Agents & LangChain create_agent: interview questions

Production questions of the kind asked in applied AI engineering interviews.

1. Larkspur's support agent cost three times as much yesterday as the day before. Traces show some runs making a dozen search_handbook calls with reworded queries before giving up. What do you check, and what do you change?

What a strong answer covers:

• Reads the ToolMessages in the long runs first: a loop that keeps rewording usually means the tool returns nothing useful, such as an index re-ingested without the answer text
• Checks what changed that day (an index rebuild, a new document type, a prompt that says "never give up") against the calls-per-run distribution
• Fixes what the model sees (data, retrieval depth, result formatting, the tool description) before touching the cap; raising the cap only buys more attempts
• Keeps a ModelCallLimitMiddleware with run_limit sized from the golden set's call counts plus headroom, and alerts on the share of runs that hit it

2. Product wants a monthly account summary for every Larkspur customer: usage, invoices, open incidents. A teammate wants to reuse the ReAct support agent. What do you build instead, and why?

What a strong answer covers:

• The steps are known in advance, so a planned pipeline (fetch usage, fetch invoices, fetch incidents, write) is cheaper, has a fixed cost per customer and runs the same way every time
• ReAct pays off when the next step depends on what a result says; a batch report has no such step
• Testing and debugging are easier when every run takes the same path; failures are recorded per step instead of steering the run
• Keeps the agent for open support questions, and considers plan first, then react, if a summary section sometimes needs follow-up
• Lesson 1 of T8.2 (T8.2, p07_advanced_agents/m26_coding_computer_use_and_deep_agents/) works through planned pipelines against the ReAct loop on applied agents

3. You replace the T5.1 baseline retriever with a ReAct agent that uses it as a tool. How do you show the change is worth it?

What a strong answer covers:

• Runs the frozen golden set through both and reports evidence recall and answer quality per question, not only the mean (in the lab, mean recall goes from 0.50 to 1.00)
• Reports what the loop costs: model calls per question (the lab's questions need 2, 3 and 4), tokens read, latency at p50 and p95
• Grades answers with a judge that is not the agent's own model, since live search results and sampling make single runs noisy
• Checks the questions the baseline already answered for regressions, such as a loop that over-searches easy questions

4. Larkspur's agent can call issue_refund behind HumanInTheLoopMiddleware. A support engineer, tired of waiting, pastes the customer's message into the same conversation again, and the next model call fails with a 400 about tool_call_ids. What happened, and how do you prevent it?

What a strong answer covers:

• The interrupt saved the AIMessage with the issue_refund call on the thread; the new user message left that call without a ToolMessage, which the provider rejects
• The only valid next step on a parked thread is Command(resume={"decisions": [...]}), with one decision per gated call
• The UI shows pending interrupts per thread_id and blocks new messages until a reviewer approves, edits or rejects
• Uses edit to fix a wrong amount instead of rejecting, and keeps reads such as an account lookup ungated so approval stays rare

5. The ticketing system needs {category, priority, refund_amount} from every Larkspur conversation. Some days the agent returns prose; on questions with no refund it writes refund_amount: 0. How do you make the output reliable?

What a strong answer covers:

• Passes a Pydantic schema as response_format on create_agent and reads structured_response, so the provider constrains the reply
• Makes refund_amount optional with a None default, so the model is not forced to invent a number; 0 and "no refund" stay distinct
• Writes field descriptions as instructions (units, allowed categories), because they reach the model as part of the schema
• Logs parse failures with include_raw=True where the model is called directly, and checks llm.profile for structured-output support before switching models