Coding, Computer-Use & Deep Agents: interview questions

Production questions of the kind asked in applied AI engineering interviews.

1. You ship a deep agent to Larkspur's support engineers, built like 03b_long_term_memory_agent.py. Someone sets USE_SANDBOX=true expecting cloud isolation, and the memory panel shows one engineer's notes in another's session. What is wrong?

What a strong answer covers:

• The script's comment promises a LangSmith sandbox, but the flag selects LocalShellBackend, which the library documents as unrestricted shell execution on the host
• FilesystemBackend(virtual_mode=True) blocks path traversal and is not a sandbox either. Code execution needs a real sandbox backend and approval on each command through interrupt_on
• Long-term memory is keyed by CURRENT_USER_ID, fixed to default-user, and every engineer reads and writes one namespace. The key must come from the authenticated user
• recall_memories matches a substring of the stored text, and a paraphrase finds nothing. The script names PostgresStore as the production replacement

2. Larkspur wants two research agents: one writes the weekly outage summary with the same sections every week, the other answers open questions from support engineers. Which gets a planned pipeline and which a ReAct loop?

What a strong answer covers:

• The weekly summary is a known-shape report. A planned pipeline has a fixed, predictable cost and runs the same steps every time, so one fixed test covers it
• The open questions are multi-hop. A ReAct agent picks each search after reading the last result and can follow a lead the plan could not foresee
• The ReAct agent's cost varies. It needs a call cap such as ModelCallLimitMiddleware and an observable goal, such as evidence for each section, instead of trusting its claim that research is complete
• The pipeline still needs a check for an empty search result per section. Many production researchers combine the two: plan first, then let each step react

3. A developer and tester pair writes a refund-proration helper for Larkspur's billing code. After one round the tester reports 100% of its tests passing and a score of 9 out of 10. How do you know the code works?

What a strong answer covers:

• A tester without a Python tool writes tests in its report and reasons about them. Its pass rate and score are the model's estimates, not executed results
• Giving the tester the Python tool makes the results real, at roughly twice the cost. Otherwise run the tests yourself, diff the changes and re-run them rather than trusting the final message
• Log per run the reflection count against the cap, the developer and tester turn counts (r + 1 and r), and whether the last code block parses
• Run model-written code in a fresh process with a timeout, closed standard input and an empty environment, and remember that this is a guard, not a sandbox

4. A vision-driven computer-use agent for Larkspur's control panel keeps clicking, never reaches its goal and raises no error. What is going on, and how do you stop it?

What a strong answer covers:

• When the observation is pixels, locating a control is the hard part. In the module's runs gpt-4o-mini missed 53 of 53 clicks even with every miss fed back
• A miss raises no error and changes nothing. The step cap is the only bound on the loop and has to be set in code
• Success is graded on the environment, as the browser agent is graded on the cart's contents, and a run that never touched the control is reported as errored, not failed
• Prefer selectors to coordinates. Where pixels are unavoidable, the stronger model in the same runs (gpt-4o, 15 misses in 26 clicks, goal met in 2 of 3 runs) costs half as much per screenshot

5. Larkspur's deep agent delegates web research to a sub-agent, and an engineer asks why the orchestrator's context stays small even after dozens of searches. Then they ask to give it eight more skills. What do you tell them?

What a strong answer covers:

• A call to task runs the sub-agent with its own messages and tools, and only its final message returns. The searches and notes stay in the sub-agent's context
• Skills use progressive disclosure: only each SKILL.md's name and description sit in the system prompt, and the model calls read_file for the full instructions when a task matches
• Eight more skills add eight more short lines to the prompt until one is needed, and each description has to say when its skill applies, because that is what the model matches a task against
• Before adding skills that run code, check the backend: execute returns an error unless the backend can run commands, and LocalShellBackend runs them on the host