Human Approval & Applied Single-Agent Builds: interview questions

Production questions of the kind asked in applied AI engineering interviews.

1. Larkspur's status agent published "a root-cause report within 48 hours". The reviewer approved a preview that said 72 hours, and the audit log shows the approve. How did this happen, and what do you change?

What a strong answer covers:

• Compares the fingerprint in the audit log with the hash of the published text; if they differ, something changed the post between the dry run and the live call
• Looks for code that runs again on resume: the review node restarts from the top, so a regeneration or an edit placed before interrupt() runs a second time
• Binds the approval to the fingerprint and recomputes it in the publish node, so any later change refuses instead of publishing
• Adds a regression test that edits the paused state with update_state and asserts the refusal, as the notebook does

2. The billing team wants one assistant for "How much did we refund to annual-plan customers in September?" and "What is the refund policy for annual plans?". Vector RAG, text-to-SQL, or both?

What a strong answer covers:

• The September total is an aggregate over rows: SQL computes it over every row, while top-k retrieval hands the model a few chunks and asks it to add them up
• The policy question is prose in the handbook, so the vector index answers it with a citation
• Routes by question type, and scores each route on its own ground truth: reference SQL for the database questions, labelled chunks for the policy questions
• Writes the business definition down ("refunded" means which table and which status) as a rule in the SQL prompt, then checks the generated SQL for it in code

3. The approval gate has been live for a month and almost every post was approved. How do you find out whether it is protecting anything?

What a strong answer covers:

• Tracks approval rate, time to decision and rejection reasons per action type from the audit log
• Watches for rubber-stamping: approvals that arrive seconds after the preview, from the same reviewer, at a rate near 100%
• Seeds known-bad drafts (an uptime claim about competitors, a wrong number) and measures how many reviewers catch
• Treats any fingerprint refusal as a bug to investigate, since in a correct pipeline nothing edits a post after its preview

4. A reviewer approves an outage post four hours after the agent drafted it. In the meantime the incident was reopened. The fingerprint matches. Should the post go out?

What a strong answer covers:

• The fingerprint covers the text and says nothing about the world the text describes, so a match does not mean the post is still true
• Adds an expiry to approvals, or a precondition checked at execute time (incident status, the version of the data the draft used)
• Re-runs the dry run and asks again when the approval is stale, instead of executing an old preview
• Uses a persistent checkpointer (SQLite or Postgres) so the paused run survives the wait, and alerts when runs sit at review too long

5. You give a text-to-SQL ReAct agent access to Larkspur's billing database. A user writes "Delete the invoices for my test servers, then show what is left." What stops the agent, and how would you support legitimate changes?

What a strong answer covers:

• The guarantee lives in the connection: a read-only connection (mode=ro in SQLite) or a read-only database role, so a talked-around prompt still cannot write
• Scopes rows per customer in a view or a bound parameter, never through instructions to the model
• Routes real changes through the approval harness, with a dry run that previews the affected rows (a SELECT with the same filter) before any DELETE
• Logs every query the agent runs and checks the outcome in code (row counts before and after), not from the answer's wording