Interview prompt
As the FDE (Forward Deployed Engineer) on this engagement, your job is to design and ship a Tool-Using Agent with Safety Controls. The agent reads email, looks things up in internal systems, edits CRM (customer relationship management) records and pays out refunds. None of that power may go unchecked. The model suggests an action; something else makes the call.
Clarify the customer problem
- Which actions can be undone, and at what amount does money need a stricter path?
- Whose identity does the agent use: the user's, a service account's, or delegated rights?
- What has to be true before an action can run with no approval?
- If the agent gets one wrong, how is it reversed, and by whom?
- What record must we keep so audits and disputes can be settled?
Users and workflows
| User | Workflow | Current pain | AI assist opportunity | Human approval needed? |
|---|---|---|---|---|
| Business user | ||||
| Approver | ||||
| Security owner | ||||
| Tool / data owner |
Requirements
Functional
Non-functional
- Latency target
- Availability target
- Cost budget
- Security/privacy constraints
- Audit/compliance requirement
Data and integration map
| Data source | Format | Owner | Freshness | Permission model | Risk |
|---|---|---|---|---|---|
Proposed architecture
Start from a rendered diagram and shape it to fit this case. Cover each part below:
- Planner and proposals
- Tool registry
- Policy decision point
- Credential broker
- Approval service
- Idempotent execution
Evaluation plan
| Metric | Good threshold | Bad threshold | Test dataset | Owner |
|---|---|---|---|---|
| Unsafe action count | 0 | Anything above zero | Review of the audit trail | Trust and safety |
| Duplicate effects | 0 | Even a single case | Logs from the idempotency layer | Reliability |
| Policy denial rate | Holds near its baseline | Jumps suddenly | Logs from the policy engine | Platform |
| Human override rate | Low and dropping | Climbing | Edits made in the review UI | Operations |
Failure modes
- An injected prompt asks for a tool the agent is not authorised to use
- The tool call works, but its response never comes back
- An approval goes out of date before the action runs
- The agent keeps retrying one action in a loop
- The credential broker is down
- A proposed refund goes over the limit set for the task
Rollout plan
- Begin with read-only triage and draft replies.
- Show that a few narrow writes can be undone.
- Turn on writes only where rollback is proven.
- Leave every money movement with a person.
- Attack the prompt with a red team.
- Release to a canary, then grant more autonomy.
Weak vs strong answer
Weak: "I'd hook the agent up to email and the CRM and let it pay refunds."
Strong: "The model may only suggest one bounded action at a time. A typed schema checks it. A deterministic policy engine then permits or rejects it. If permitted, a broker issues a scoped credential that expires quickly, and an idempotent gateway makes sure the effect happens exactly once. Anything a tool returns is untrusted data, never instructions."
Candidate scorecard
Final 2-minute spoken answer
Rehearse this out loud before you look at it. Summarising the whole design in two minutes is its own skill, and it is usually the last thing you are asked for.