Interview prompt
As the FDE (Forward Deployed Engineer), design a GenAI system for Customer Support Automation. Routine requests should be handled end to end. Risky ones go to a human, who receives the full context. The two harms to prevent are an action nobody authorised and a wrong answer given with confidence.
Clarify the customer problem
- Which channels are in scope, and what are the normal and peak volumes?
- How do we confirm who the customer is before any account tool runs?
- What may the system do on its own, and what needs sign-off?
- At handoff, what must the human agent see, and within what time?
- How many languages do we support, and what if we cannot tell the language?
Users and workflows
| User | Workflow | Current pain | AI assist opportunity | Human approval needed? |
|---|---|---|---|---|
| Customer | ||||
| Support agent | ||||
| Support operations lead | ||||
| Security / legal reviewer |
Requirements
Functional
Non-functional
- Latency target
- Availability target
- Cost budget
- Security/privacy constraints
- Audit/compliance requirement
Data and integration map
| Data source | Format | Owner | Freshness | Permission model | Risk |
|---|---|---|---|---|---|
Proposed architecture
Pick a rendered diagram as your starting point and adapt it to this case. Cover each part below:
- Identity verification
- Intent and risk routing
- Grounded retrieval
- Tool policy gateway
- Escalation
- Monitoring
Evaluation plan
| Metric | Good threshold | Bad threshold | Test dataset | Owner |
|---|---|---|---|---|
| Safe automation rate | Holds steady for each intent | Drops for any intent | Labels from QA review | Support operations |
| Incorrect resolutions | At or under baseline | Over baseline | Tickets that get reopened | Support QA |
| Repeat-contact rate | Level or falling | Climbs after a change | Contacts linked in the CRM | Support analytics |
| Cost per resolved case | Goes down | Rework cancels the saving | Finance reports | Finance operations |
Failure modes
- A policy answer is wrong but sounds certain
- Customer text carries a prompt injection
- An action runs before identity is checked
- A refund goes through but the reply times out
- Out-of-date account data leads to a promise
- The handoff drops context, so the agent starts over
Rollout plan
- Assist agents only; a human sends every reply.
- Automate intents that are low-risk and easy to reverse.
- Introduce tool classes one at a time.
- Review a sample of automated cases.
- Keep a kill switch for each intent.
- Roll back one intent, not the whole system.
Weak vs strong answer
Weak: "I'd pass every message to the model and let it close the ticket."
Strong: "I'd treat this as a decision pipeline with routing, not a chatbot. Identity is checked before any account tool. Each request gets an intent and a risk tag. Drafts rest on approved policy. Then a deterministic policy gateway, not the model, chooses between auto-resolve, approval and escalation."
Candidate scorecard
Final 2-minute spoken answer
Rehearse this out loud before you look at it. Summarising the whole design in two minutes is its own skill, and it is usually the last thing you are asked for.