Customer Support Automation

Support process is a customer has a problem, someone answers or acts, and we close the case without making money or data worse.

G02 covers one slice: chat/email/voice in, then answer, get approval, or hand off. Not a free-form company agent.

End to end, as a double-charge ticket:

1. Message lands — “You billed me twice.”
2. We verify it’s them before any account tool.
3. We look up live billing and the refund policy.
4. A model drafts a reply or a refund proposal. It does not pay anyone yet.
5. A policy gate checks identity, amount, freshness, duplicates.
6. Safe FAQ goes out. Money waits for a human. Unclear cases get a full handoff pack.

That’s it: verify → facts → draft → gate → answer, approve, or escalate. Lawsuits and open negotiation stay human.

Core idea: Build a routed decision pipeline that knows when to answer, when to ask for approval, and when to hand off. The model can draft and recommend; verified identity and a deterministic policy gateway decide whether an account action may happen.

The unchanged source case is the full reference. Use this guide for the spoken design, the Deep Dive for follow-ups, and the Cheat Sheet for last-minute recall.

“I want to reduce handling time and cost without increasing incorrect or harmful resolutions. I’ll identify which requests can be answered, which need approval, and which need a human. Then I’ll design the identity, policy, action, and handoff boundaries around those decisions.”

Use a double-charge request as the running example. A fluent reply is not enough: the system must verify the customer, check live billing facts, apply the refund policy, avoid a duplicate refund, and hand off cleanly if anything is uncertain.

Questions to ask the interviewer

These questions set the first-release boundary. The source case, §1 has the complete discovery table.

If the interviewer cannot provide numbers, say which assumptions you are making. Keep money-moving and account-changing actions gated until identity, authority, and policy are clear.

The anchor for its support-automation variants

Learn the shared decision pipeline here, then change the boundary that dominates each variant.

The easiest way to frame requirements in an interview is:

Functional = what the system does. Non-functional = how well it does it and what constraints it must satisfy.

Functional requirements — what the system must do

1. Normalise intake — chat, email, web, or voice transcripts into one case envelope.
2. Classify intent and risk — before choosing an answer or action path.
3. Verify identity — before account data or account tools.
4. Retrieve live facts — approved policy and current customer/order state; draft from evidence.
5. Gate proposed actions — authorisation, validation, freshness, threshold, and approval.
6. Escalate with context — conversation, evidence, attempted actions, and why automation stopped.
7. Audit the case — decisions, tool calls, overrides, and outcomes.

Non-functional requirements — how well / under what constraints

First release

Exclude autonomous legal complaints, open-ended negotiation, unsupervised refunds or cancellations, cross-system repair without review, and ambiguous proactive outreach.

Interview shortcut

If asked “What are the requirements?”, say:

“Functionally, I need to verify the customer, pull live facts, draft from evidence, and gate any account action. Non-functionally, the constraints are latency by risk tier, fail-closed money moves, audit, and net value rather than deflection.”

This simplifies the source architecture, §4 for a whiteboard explanation.

Step-by-step architecture

• Step 1. Receive the request. The channel gateway normalises chat, email, web, or voice into a support case.
• Step 2. Verify the customer. Check identity and assurance; failed verification permits only generic help or handoff, not account actions.
• Step 3. Route by intent and risk. Rules or a small model classify the request; only a multi-system case invokes a bounded support-agent planner.
• Step 4. Gather facts. Retrieve approved policy and live account facts; the LLM (large language model) response generator then drafts a reply or proposed tool action.
• Step 5. Gate the proposal. The deterministic policy gate checks identity, allowed tools, thresholds, freshness, and required approval.
• Step 6. Choose the outcome. Return a safe answer, send a case with context to a human, or obtain approval for an action that needs it.
• Step 7. Execute and record. Send permitted actions through the scoped tool gateway to the system of record; audit the decision and outcome.

Model and agent role: The LLM drafts and proposes; a bounded agent planner coordinates only requests that truly span systems. Simple lookups use direct routes. Neither the LLM nor the planner authorises an account action; the deterministic gateway and human approval path do that.

Three boundaries to point at:

• Identity: pre-auth FAQ can use a limited generic mode; account tools require verified assurance.
• Authorisation: the model proposes; the tool policy gateway checks identity, allowed action, thresholds, freshness, and human approval. Confidence never grants permission.
• Sync/async: accept and route promptly; run permitted longer actions in the background with status and reconciliation.

The automation layer owns the case workflow and evidence of its decisions. CRM (customer relationship management), identity, order, and billing systems remain authoritative for customer facts and money. Policy articles are versioned; live account or billing state is fetched before an action.

Use a planner only when the request truly spans systems, such as refunding an order, cancelling a warranty, notifying shipping, and creating a ticket. A simple “track my order” request should take the direct lookup path. Keep session context, cross-session customer history, and enterprise knowledge as separate memory layers; they have different freshness and privacy rules.

Keep a versioned Case, an immutable ProposedAction, and a Handoff. Every write boundary needs an idempotency key; the action record can use an argument hash to recognise a duplicate without storing raw sensitive arguments. Check Case.version so automation cannot overwrite a human’s newer decision. A duplicate customer message should return the existing case, not create a second one.

A useful handoff contains: original message; identity status; parsed intent and risk; retrieved evidence; policy checks; tool outputs and error codes; attempted or pending actions; and the exact reason automation stopped. Prioritise the human queue by risk and impact. A transcript alone makes the agent repeat work and hides the failed control.

• Identity unavailable: account tools fail closed; offer only limited generic help or escalate.
• Policy retrieval unavailable or stale: do not present a superseded rule as current; withhold the commitment and hand off.
• Billing lookup times out: block the refund because facts are incomplete; attach the failure and conversation to the handoff.
• Refund response times out: treat execution as uncertain; reconcile against the ledger/action record before any retry.
• Model endpoint fails: circuit-break, queue eligible work, and route risky cases to humans.
• Language detection fails: use a language-neutral intake and escalate when uncertain.
• Confident but wrong policy answer: stop automated delivery, preserve prompt/output/policy version, hand off, and add a regression case.

Retries are bounded and jittered, failed jobs remain inspectable, and timeouts never convert a blocked action into an allowed one.

The source’s sizing exercise assumes 2M tickets/month, 100 QPS (queries per second) peak, and 20 languages. With an illustrative 70% routine / 20% ambiguous / 10% escalated mix and 2 / 3 / 1 model calls respectively, that is about 4.2M model calls/month, before retries and moderation. The same assumptions imply roughly 2.2M–2.4M retrieval calls/month. Treat these as interview assumptions, not measured production demand.

The business check is NetValue = time saved − model/retrieval cost − wrong-resolution cost − repeat-contact cost. Lower model spend is not a win if it creates expensive recontacts.

Safe automation rate means automated cases resolved without correction, complaint, or harmful escalation divided by automated cases. Read it beside incorrect-resolution rate, first-contact resolution, repeat contact, CSAT (customer satisfaction score), handling time, and cost per resolved case. Deflection alone only tells you that work moved.

Use four views: technical health (latency, tools, queues, fallback); model quality (grounding, retrieval, escalation correctness, unsupported answers); adoption (agent use, acceptance, overrides); business outcome (resolution, CSAT, recontact, cost). Segment by intent, language, and customer group. A falling handle time with rising repeat contact signals rework, not improvement.

Before launch, require stable retrieval on target intents, representative human QA (quality assurance) approval, correct fallbacks, complete logging, and an exercised rollback path. Support operations owns escalation and adoption; engineering owns reliability and guardrails; product owns the gate; security/legal own access and action permissions.

1. Agent-assist: draft and summarise; a human sends every reply.
2. Prove the controls: use real traffic and historical tickets to validate retrieval, risk routing, handoff completeness, and audit.
3. Narrow automation: enable high-volume, low-harm, reversible intents such as order status or shipping ETA (estimated time of arrival).
4. Add one tool class at a time: ticket tagging, read-only lookup, then increasingly sensitive actions with explicit approval gates.
5. Canary and review: sample automated outcomes; keep a kill switch per intent and per tool; roll back the drifting slice.

When asked for “80% autonomy in six months,” first ask 80% of what—tickets, intents, or cases resolved without recontact? Let safe automation and repeat-contact results determine expansion, especially for money-moving and legal requests.

• Multi-tenant SaaS: carry tenant identity through retrieval, tools, cache, and logs; test the same question as users from different tenants. Release gate: zero permission violations in the tenant ACL (access control list) red-team suite.
• ServiceNow/ITSM: classify priority and assignment from ticket, asset/CMDB, and approved knowledge data. Propose configuration changes through the customer’s ITIL change process; do not bypass it.
• Multilingual: language detection needs a default path; do not silently drop an unknown language.
• Framework details: durable approval interrupts belong at the risky tool, not the start of the turn; bounded retries and agent steps prevent loops; hybrid retrieval and source-specific indexes protect exact IDs and error codes. See the Deep Dive for the source’s six concrete mock designs.

Spend the hour roughly as follows: opening and requirements (10 min), architecture and control boundaries (15), action safety/handoff/failure drill (15), evaluation and rollout (10), scale/cost/variants and close (10).

90-second answer:

“I would design support automation as a routed decision pipeline. The gateway normalises the message, identity verification establishes what account data and tools are available, and an intent router tags risk. Approved policy and current account facts ground a draft or proposed action. A deterministic gateway checks authorisation, freshness, thresholds, and whether a human must approve; the model cannot grant itself permission. Simple, low-risk requests can be answered quickly. Money-moving or ambiguous cases go to an agent with evidence, attempted actions, and a reason. Every write is idempotent, and an uncertain side effect is reconciled before retry. I’d launch in agent-assist, then automate narrow reversible intents, adding tools one class at a time with per-intent rollback. I’d judge it by safe automation, incorrect resolutions, repeat contact, CSAT, latency, and cost per resolved case.”

Interviewer trigger → short answer

Final mental model: Verify → Classify risk → Fetch authoritative facts → Draft → Policy gate → Answer / approve / hand off → Audit and learn.

This is the technical backup for the Main Interview Guide. The unchanged source case contains the full synthesis and references.

Model and agent roles: Rules or a small model route intent and risk. The LLM (large language model) response generator drafts a reply or proposed action; a bounded planner is reserved for multi-system requests. The policy gateway and approver, not the model, authorise effects.

The automation service owns workflow state and decision evidence, not customer truth. A stale shipping event may make an answer wrong; a stale ledger or assurance level may make an action harmful.

• Case(id, customer_id, channel, intent, risk, state, version) moves through new, triaged, waiting_approval, escalated, and resolved. version supports optimistic concurrency when a human and automation race.
• ProposedAction(id, case_id, tool, args_hash, decision) is immutable after its outcome. args_hash identifies a logically duplicate request without persisting raw sensitive arguments.
• Handoff(case_id, summary, evidence_refs, attempted_actions) packages the case for human takeover; include identity status, checks, error codes, and the precise stop reason.

Keep the API small: POST /v1/support/messages, POST /v1/cases/{id}/actions, POST /v1/cases/{id}/escalate, and POST /v1/cases/{id}/quality-review. Authenticate and tenant-scope each request; validate schemas and define idempotency for every write. Version payload and endpoint contracts independently of risk-policy changes.

Duplicate message drill: if the client resends Idempotency-Key: msg_9f1c after a timeout, return the original case_123; do not create a second case. Refund timeout drill: a timeout is an unknown outcome, not permission to repeat the payment call. Check the action record or ledger, then confirm, keep pending, or escalate.

Use 409 for a state transition no longer allowed and 422 for typed validation failure, as in the source example. Order queues by conversation, customer, or account ID to avoid out-of-order replies and handoffs.

Control plane: risk tags, approval/refund thresholds, workflow tool allowlists, confidence thresholds, policy/model/prompt versions, routing weights, low-risk intent list, per-intent and per-tool kill switches.

Data plane: channel ingress; identity and assurance; intent/risk routing; knowledge and live account retrieval; draft/proposal generation; policy gateway; calibrated outcome; approved action execution or human handoff; trace and evaluation events.

The boundaries are deliberately narrow:

1. Untrusted text: customer messages, retrieved articles, and copied tickets are data. Prompt injection cannot become a tool instruction or policy change.
2. Identity: pre-auth traffic stays in a limited generic mode. Account-specific data and tools require an approved identity path and sufficient assurance.
3. Policy gateway: checks action allowlist, identity assurance, current facts, value threshold, account/tenant scope, and review rule on each proposal. The model’s confidence is never authorisation.
4. Tool service: scoped credentials, typed arguments, idempotency key, immutable record, and reconciliation on uncertain outcomes.
5. Human approval: the interrupt is placed at the risky tool call so read-only work does not wait needlessly.

Contain blast radius by tenant, region, workflow, and dependency. A refund-tool incident should not disable FAQ answers. A language router outage should not erase ticketing or audit.

Classify informational, account-sensitive, money-moving, legal-sensitive, and safety-sensitive requests. A safe direct path serves a single FAQ or order lookup. A planner earns its latency and cost for multi-system work, such as refund + warranty cancellation + shipping notice + priority ticket. Keep RAG (retrieval-augmented generation), tools, and memory as selectable capabilities rather than forcing every request through all three.

Separate memory by lifetime and authority:

Rules can tag simple cases. Small models can handle classification and routine FAQs; stronger reasoning can be reserved for ambiguous cases after retrieval and policy filtering. Do not globally shrink the model or inflate context. Inspect failed examples, then change the relevant route, evidence set, or model.

The source’s illustrative requirements are routine p95 <3 s, ambiguous p95 <8 s, and high-risk handoff bundle p95 <15 s. The last tier does not authorise automated final decisions. A sample 2M-ticket/month, 100-QPS-peak, 20-language workload yields about 67k tickets/day on average, but bursts drive gateway, queue, and escalation capacity.

With a 70/20/10 routine/ambiguous/escalated split and 2/3/1 model calls: 1.4M × 2 + 0.4M × 3 + 0.2M × 1 = 4.2M monthly model calls before retries, moderation, and tools. Retrieval is roughly 2.2M–2.4M calls on the source assumptions. These are planning calculations, not measured traffic.

At 10× conversations, use stateless gateways and orchestration, retrieval partitions by product/region/language/business unit, regional deployment where residency requires it, asynchronous long-running actions, rate limits by user/tenant/tool, and circuit breakers. Isolate offline summarisation and human escalation from routine classification. A semantic FAQ cache near ingress can avoid expensive orchestration only when tenant, permission, and content version are in the key. Identity assertions, account status, authorisation decisions, and money state require live authority.

Cost is a workflow calculation: NetValue = V_time_saved − C_model − C_wrong_resolution − C_recontact (and account for retrieval/operations when pricing it). Measure cost per resolved issue and by intent. A shorter cheap answer that drives recontact can lose money.

For a wrong policy answer, ask whether a side effect occurred or a customer was misled. Preserve prompt, output, evidence, and policy version; stop automated delivery; hand off; add a regression example. For a billing timeout before action, the gateway blocks a refund because authoritative facts are missing, then hands off. For a timeout after a refund call, check the ledger before deciding whether anything remains to do.

Policy retrieval cannot be replaced by an unmarked stale answer. Model failures circuit-break and queue eligible work. Use short deadlines, bounded jittered retries, and dead-letter queues so failures stay inspectable. Explain the handoff at workflow level: which intent and rule fired, which fact was absent or contradictory, and which action was attempted.

Safe automation rate counts automated cases resolved without correction, complaint, or harmful escalation, divided by all automated cases. Use ticketing and QA (quality assurance) labels; support operations and engineering inspect drops by intent or segment. An incorrect-resolution rate above an intent-specific baseline, rising recontact, or falling CSAT can veto a better deflection number.

The gate requires stable retrieval on target intents, representative human QA approval, correct fallback, complete logging, and exercised rollback. Product owns go/no-go; support operations owns escalation and adoption; engineering owns reliability and guardrails; security/legal own access and tool permissions; frontline managers own training and review.

Rollout: agent-assist first; prove retrieval, routing, handoff, and audit; automate narrow low-risk intents; enable one tool class at a time; canary and sample outcomes; keep an intent/tool kill switch. Revert a drifting intent to agent-assist without taking down healthy workflows. Separate tenant configuration (thresholds, languages, retention) from adapters (CRM, ticketing, identity, KB) and shared services (policy, audit, retrieval, observability).

Multi-tenant SaaS support: Tenant, Entitlement, and RetrievalACL data constrain every retrieval and tool call. Test the same question under different tenants and roles. Look for cross-tenant evidence, wrong entitlement advice, exposed internal logs, and credits without approval. A red-team ACL (access control list) suite has a zero-violation release gate.

ServiceNow/ITSM: classify category and priority, retrieve an approved KB article/runbook, request missing information, suggest assignment, and apply workflow actions only within policy. The Ticket, CMDB (configuration management database) Asset, KBArticle, AssignmentRule, and WorkflowAction records let the system explain a decision. Risks include wrong priority/group, premature closure, unauthorised access changes, requester PII (personally identifiable information), and injection in ticket text. Configuration-item changes follow the customer’s existing ITIL change process; the agent proposes a change record.

These are example implementations, not mandatory product choices:

1. Multilingual triage (#95): detect language → classify → route → draft → translate back. An intent branch needs a default arm. Example p95 <3 s allocation: 200 ms detection, 300 ms classification, 1.5 s streamed drafting, 500 ms translation (2.5 s total). Bound retries and provide a classifier fallback.
2. Human-gated graph (#96): structured {category, risk_level} output; interrupt inside refund/deletion tool; durable Postgres checkpoint keyed by ticket; idempotent side effect keyed by ticket and action; concurrent read-only lookups; at most three self-correction iterations before escalation.
3. Agentic RAG (#97): separate product-doc and runbook indexes, each hybrid BM25/vector for exact error codes; cheap router; retrieve 20/source concurrently, rerank merged candidates to five when scores are close; 400 ms retrieval budget; cite source and abstain below relevance threshold; weekly labelled recall@k gate.
4. Growing support search (#99): wiki, ticket, and release-note sources with per-source hybrid weights; incremental append and source metadata; coverage-gap grading and human fallback for new error codes; cross-encoder rerank of fused top 20 under a 1.5 s p95 target; track fallback rate.
5. Travel booking (#101): supervisor with read-only booking lookup, RAG policy QA, and one write-capable change/cancel specialist. Compute refund amount with pure logic; example >$200 approval interrupt; idempotency by booking and request ID; durable conversation checkpoint, separate customer memory, 15-step and cost ceilings; direct workflow for simple baggage FAQ.
6. Parent-document retrieval (#102): example child chunks of about 400 characters and parents of about 2,000 (LangChain's example sizes; tune on your data); shared Postgres vector/doc store, HNSW once exact scans start to eat the latency budget, idempotent daily append keyed by article ID and modification time; p95 <2 s budget dominated by streamed generation; cap final k at 3–4 and abstain below similarity threshold.

Keep these mock-specific thresholds attached to their scenarios. They are not global G02 requirements.

• “80% autonomous in six months?” Define denominator and safe resolution window. Expand by intent only while incorrect resolutions, recontact, and harm remain within gates.
• “Confident but wrong?” External evidence, policy, live state, and gateway rules can block a confident model. Preserve the record and hand off.
• “Duplicate refund?” Idempotent action, immutable record, state check, ledger reconciliation after timeout.
• “18-second prototype?” Profile auth, retrieval, rerank, generation, tools, and verification; route simple intents directly, parallelise independent reads, stream where useful, run permitted actions async. Preserve high-risk checks.
• “Prompt injection in a ticket?” Ticket text is data; tools and retrieval enforce scope; test malicious instructions in documents, tickets, emails, and tenant data.
• “Launch next Monday?” Controlled shadow/draft pilot and narrow reversible actions with rollback, rather than broad unsupervised release.

For a 60-minute round, give the outcome and constraints first, draw the gates, spend the most time on identity, action policy, handoff, failure recovery, and rollout, then discuss scale and cost. The source’s 90-second and two-minute speeches are useful examples; keep your own explanation conversational.

The source case retains the detailed comparison table, original interview scripts, check-yourself prompts, and references to casebook, FDE (Forward Deployed Engineer), OpenAI Applied, and study-guide material. This Deep Dive relocates their mechanisms and scenario-specific figures; consult the source when quoting a particular mock or original reference path.

Use the Main guide for the spoken answer and the Deep Dive for technical follow-ups. The source case remains the full reference.

A support assistant is a routed decision pipeline: the model suggests, the policy gateway decides, and uncertain or risky work reaches a human with context.

Channel → Verify identity → Classify intent/risk → Fetch approved policy + live facts → Draft/propose → Deterministic policy gate → Answer / approval / handoff → Audit.

LLM/agent role: Rules or a small model route; the LLM (large language model) drafts; a bounded support-agent planner handles multi-system cases. The policy gateway decides whether any tool action runs.

1. Verify identity before account data or account tools.
2. Confidence is not permission; the gateway checks allowlist, threshold, freshness, and approval.
3. Billing, orders, CRM (customer relationship management), and identity remain systems of record.
4. Every write is idempotent; reconcile an uncertain side effect before retry.
5. Optimise safe resolution, not raw deflection.

Functional: route by intent and risk; retrieve grounded facts; draft; gate actions; hand off fully; audit outcomes.

Non-functional: routine p95 <3 s, ambiguous <8 s, high-risk handoff <15 s in the source example; secure account access; safe degradation; immutable audit; cost per resolved case.

Illustrative load: 2M tickets/month, 100 QPS (queries per second) peak, 20 languages. Under the source’s traffic and call-count assumptions: ~4.2M model calls/month and ~2.2M–2.4M retrieval calls/month.

Original message + identity status + intent/risk + evidence + policy checks + tool outputs/errors + attempted actions + why automation stopped.

• Wrong policy answer: stop delivery, preserve evidence and versions, hand off, add regression.
• Billing lookup timeout: block action, hand off with missing-fact reason.
• Refund timeout: inspect action record/ledger before doing anything again.
• Model or language route outage: narrow safe path or human queue.

Gate: representative QA (quality assurance), correct fallback, complete logging, stable retrieval, exercised rollback. Track safe automation, incorrect resolutions, first-contact resolution, repeat contact, CSAT (customer satisfaction score), p95, and cost per resolved case.

Sequence: agent-assist → narrow reversible intents → one tool class at a time → canary and sample → kill switch per intent/tool.

• Multi-tenant: tenant predicate on every retrieval and tool call; zero ACL (access control list) red-team violations.
• ServiceNow: priority/assignment + approved KB/CMDB context; configuration changes through existing ITIL process.
• Complex agent: planner only for multi-system requests; direct path for simple lookups.

Measure by intent → route by risk → bound calls, steps, tokens, and retrieval → cache safe FAQ traffic. Keep account facts live. Check time saved − model/retrieval cost − wrong-resolution cost − recontact cost.

Close: Reduce handling time without increasing harmful mistakes; prove it with a gated rollout and customer outcomes.

Resolve the routine ticket fast enough to matter, without letting a fluent model take an action or make a promise it was never authorised to make.

The product is not a chatbot that answers everything. It is a routed service that must decide when to act, when to ask for approval, and when to stop. A fluent model is one input to that decision, never its authority. This page consolidates the group's prompts into one read for the day before. Everything else in the group is a delta on it.

Open with the harm case, not the architecture. The ask sounds like "automate routine support", and the single reframe interviewers listen for is that the product must decide when to act, when to ask, and when to stop. Say it in the first minute, because it sets up every gate that follows.

"I'd like to optimise for the outcome first: reduce handling time and cost without increasing incorrect or harmful resolutions. I'll clarify the highest-risk intents, estimate scale, sketch the control flow, and then spend most of my time on escalation, safety and rollout."

The running example is a customer who says "I was charged twice for last month's subscription and need this fixed today." The naive design sends that to a model, drafts a reply and resolves the ticket. The real design starts earlier. Verify who the user is. Determine whether the request touches money. Decide whether it is safe to act on the account. Preserve enough context for a human to take over if automation hesitates. If the billing provider is down, or the model sounds confident but the policy check fails, degrade into a safe handoff rather than force a bad answer.

The three answer tiers show what the reframe buys. The weak answer sends each message to an LLM (large language model), drafts a reply and resolves the ticket. It treats model output as truth, never verifies identity before touching an account, and puts no policy gate between a fluent answer and a real refund. The average answer classifies intent, retrieves policy and account context, drafts a grounded reply and escalates on low confidence. It never names which actions need approval, never says what happens when a refund executes but the response times out, and treats confidence as sufficient grounds to act. The strong answer designs a routed decision pipeline. Identity is verified before any account tool. The router tags risk. Retrieval grounds the draft. A deterministic tool policy gateway, not the model, decides whether an action may execute. Latency is tiered by risk. Dependency failures degrade into a handoff carrying full context. The launch is agent-assist only, then reversible intents, then one tool class at a time with a kill switch per intent.

Ask the questions that move risk boundaries. Name the six discovery areas fast and unprompted: channel and volume, action authority, identity and data sensitivity, freshness of knowledge, escalation and human-in-the-loop, multilingual handling.

Interviewers often answer only half of these on purpose, to see whether the candidate can assume without gambling on the highest-risk area. Safe defaults exist. Chat and email but no phone means a text-based model. An unknown language mix means the dominant language plus a fallback escalation path. Unknown volume means scalable but not over-engineered. Dangerous assumptions weaken the control boundary. They are that identity is always verified, that any request can be automated, that the model's answer is inherently safe, and that the CRM (customer relationship management) is always the source of truth. When in doubt, protect the highest-risk constraint first: no unauthorised action and no confidently wrong answer on sensitive topics.

Map the people, because each notices a different failure first.

Declare the scope fence before the first box, because without a non-goals list the design expands until risk dominates the interview. First-release exclusions:

• open-ended negotiation with customers
• autonomous handling of legal complaints
• fully unsupervised refunds or cancellations
• cross-system data repair with no human review
• proactive outreach on ambiguous policy violations

A constraint is a boundary the design must obey; a preference is polish. "Refunds above a stated threshold require human approval" is a constraint and shapes trust boundaries, authorisation and audit. "Show recent purchases on the right side of the console" is a preference and decides nothing about admissibility. State every requirement so a test can fail it.

The functional must-haves form the pipeline in order. Classify and route every request by intent and risk before any retrieval or drafting begins. Retrieve grounded customer and policy context from CRM, order and policy sources before drafting. Draft or send answers gated by confidence and risk level. Execute tools such as refunds, address changes and subscription actions only through a controlled, authorised layer. Hand off to a human with full conversation and action history whenever automation hesitates. Record every retrieval, decision, tool call and human override so any outcome is attributable afterward.

The non-functional requirements tier by risk rather than one flat SLO (service level objective).

Every requirement has an owning component, and the traceability table is the proof.

Size for the load that breaks the system, not the average. The practice numbers are 2 million tickets a month, 100 QPS peak and 20 languages. Two million a month is about 67,000 a day, but a morning spike after a product incident or a billing change arrives far above that. At 100 QPS the control plane cannot be a single-threaded pipeline. Twenty languages make English-only assumptions unsafe. Turn volume into calls, because tickets times one is not the model bill. Assume 70 percent routine, 20 percent ambiguous and 10 percent escalated. Routine is 1.4M tickets times 2 calls, or 2.8M. Ambiguous is 0.4M times 3, or 1.2M. Escalated is 0.2M times 1, or 0.2M. That is about 4.2M model calls a month, plus retries, moderation and tools. Retrieval runs 1.4M plus 0.8M plus up to 0.2M, so 2.2M to 2.4M calls a month. Retrieval therefore dominates freshness logic, cache design and failure isolation; it is not a cheap helper. Handled only by people, a queue this size needs well over a thousand full-time agents (assuming ~1,500 tickets per agent a month). Deflecting 30 to 50 percent of tickets (a conservative planning assumption) shows up as delayed hiring, less overtime and agents moved to complex cases, not as half a team eliminated.

The NetValue equation is the most reusable move on the board. Start with time saved. Subtract model and retrieval cost. Subtract the cost of a confidently wrong resolution, which I'd expect to outweigh the model bill in most support settings. Subtract the cost of a repeat contact. A system that saves ten seconds per ticket and raises recontact is not a win. At 10× growth, revisit partitioning rather than server count: split online answering from offline summarisation, isolate escalation traffic from routine classification, and move long-running enrichment out of the critical path. Peak concurrency and escalation rate control the architecture more than the exact monthly count.

The automation layer owns workflow state and the evidence for its decisions. It never owns the customer's truth. The CRM (customer relationship management), billing, order and identity systems remain the systems of record, and if the source says an account is locked no cache may say otherwise.

The three core records carry the workflow. Case(id, customer_id, channel, intent, risk, state) is the live object, with lifecycle new, triaged, waiting_approval, escalated, resolved, kept hot while the support window is open. ProposedAction(id, case_id, tool, args_hash, decision) is the proposed or executed step. args_hash deduplicates logically identical requests without storing raw sensitive arguments, which is where idempotency becomes real. The record is immutable once its outcome is recorded. Handoff(case_id, summary, evidence_refs, attempted_actions) is what a human receives when the machine stops, and its job is to make the agent faster and safer. Case.version supports optimistic concurrency, so a race between automation and a human fails cleanly instead of overwriting.

Every write boundary carries an idempotency key. A chat client sends POST /v1/support/messages twice with Idempotency-Key: msg_9f1c because the first response timed out. The first request creates case_123 and returns {"case_id": "case_123", "state": "triaged", "risk": "medium", "next_action": "draft_for_agent"}. The second must not create case_124; it returns the same case_123 and logs that the duplicate was deduplicated at the write boundary. The API surface stays small and boring: messages, POST /v1/cases/{id}/actions, POST /v1/cases/{id}/escalate, POST /v1/cases/{id}/quality-review, each with an authenticated caller, tenant scoping, schema validation and idempotency semantics. Payload schemas and endpoint contracts are versioned separately, so a case written at one version stays readable after the policy engine learns a new risk label.

One diagram carries the design, and each component exists because the one before it needs a narrow next step. The organising idea is the split. The data plane carries the conversation, the facts and the draft through the low-latency path. The control plane decides whether that path may continue, whether extra checks are needed and whether a human must intervene. In my view, most escalation failures are control decisions made too late or in the wrong place, not generation failures.

 ╔═══════════════════════════ CONTROL PLANE (decides whether the path may continue) ═══════════════════════════╗
 ║  risk tags per intent · refund threshold · tool allowlist per workflow · confidence thresholds · policy version ║
 ║  prompt + model versions · routing weights · low-risk intent list · kill switch per intent and per tool          ║
 ╚═════════════════════════════════════════════╤═══════════════════════════════════════════════════════════════════╝
                                               │ configures every gate below
 ╔═══════════════════════════ DATA PLANE (carries the conversation, facts and draft) ═════════════════════════════╗
 ║                                                                                                                  ║
 ║  INGRESS — sync                        IDENTITY — sync when account data or action is possible                   ║
 ║   chat · email · web · voice ─> omnichannel gateway ─> identity verification ─> assurance level                  ║
 ║                                 (one envelope,           (consults auth system      │ fail: limited generic mode ║
 ║                                  owns no state)            of record)               v        or escalate         ║
 ║                                                                                                                  ║
 ║  DECISION — sync, ordered by risk                                                                                ║
 ║   intent + risk router ─> knowledge retrieval ─> response generator ─> TOOL POLICY GATEWAY ─> confidence        ║
 ║   informational ·         policy articles ·      draft reply and/or    deterministic:          calibrator        ║
 ║   account-sensitive ·     account facts ·        proposed tool call    assurance ok? action     ┌─ auto-resolve  ║
 ║   money-moving ·          prior cases            (a recommendation,    allowed? refund over     ├─ request       ║
 ║   legal · safety          (live before action)    never a decision)     threshold? facts fresh?  │  approval      ║
 ║                                                                         ── the authorization    └─ escalate     ║
 ║                                                                            boundary                    │         ║
 ║  ACTION — async where the side effect allows                                                           v         ║
 ║   tool execution service (scoped credentials, typed args, idempotency key) ─> CRM · billing · orders · tickets  ║
 ║   refunds, warranty, shipping investigations run in the background; "your request has been submitted" now      ║
 ║                                                                                                                  ║
 ║  HUMAN — operational boundary                                                                                    ║
 ║   human-agent queue (prioritised by risk and impact) <── Handoff(summary, evidence_refs, attempted_actions)     ║
 ║   manager approval queue for gated actions                                                                       ║
 ║                                                                                                                  ║
 ║  OBSERVABILITY — every stage writes                                                                              ║
 ║   quality evaluation store (solved · reopened · corrected · overridden) ─> calibration, routing, policy updates  ║
 ║   traces (prompt + model + policy version, tool calls, latency, cost) ─> four-layer dashboard                    ║
 ╚══════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝

The same flow as a rendered diagram, for viewers that draw Mermaid:

Read the components in dependency order, because that is the order they exist and the order they fail.

Three boundaries are worth pointing at while the diagram is up. The identity boundary sits before the router, so pre-auth traffic can only ever reach a limited generic mode. The authorisation boundary is the tool policy gateway: everything to its left may suggest, nothing to its left may act. And the sync/async boundary sits at the action layer. A refund, warranty cancellation or shipping investigation runs in the background while the customer hears "your request has been submitted" immediately. Partition every queue by conversation, customer or account ID, because without an ordering key duplicate replies and out-of-order handoffs multiply under load. Cache policy articles, language detection results and non-authoritative summaries; never cache identity assertions, money movements, account status or authorisation decisions.

"The customer sounds right" is not authentication. Once a workflow reaches billing, PII (personally identifiable information), address changes, password resets or account access, identity is confirmed through the customer's approved path before the tool layer can act. Verification returns an assurance level, and the gateway checks it on every action. Low-risk pre-auth FAQ traffic stays in a limited generic mode with no account tools at all.

Security here is a chain, not a check, and the threat model has four seams. Customer text can carry prompt injection, malformed account numbers, copied ticket history, hostile instructions or social engineering. It belongs in a message channel and is never treated as a privileged instruction. Tools are scoped by workflow with least privilege: an assistant that only needs order status is never handed refund, cancellation or payment-history power. Identity is verified before account tools, as above. And an immutable audit records the whole decision: who requested the action, which model version suggested it, which tool was invoked, what fields were validated, which policy version was in force, and whether a human reviewed or overrode it. The evidence survives the incident.

Blast radius is defined in layers so one failure stays contained:

• by tenant, so a bad policy prompt does not disable support for every customer
• by region, so a queue outage does not corrupt the global audit log
• by workflow, so a refund-tool failure does not affect FAQs
• by dependency, so a language-routing outage does not take down ticketing

Any one control can fail, which is why the chain matters.

Model output is not truth. It is one input to a controlled workflow. Every place the system can take an action, reveal account data or assert a policy needs a guardrail around identity, authorisation, validation and recovery. The router tags each case informational, account-sensitive, money-moving, legal-sensitive or safety-sensitive. The generator produces a draft or a proposed tool call, still a recommendation. The tool policy gateway then applies deterministic checks. Is identity assurance sufficient? Is the requested action allowed? Does the refund exceed the threshold? Does the case require human review? Is the data accessible to this agent tier? The calibrator chooses among three outcomes. High confidence plus policy pass auto-resolves. A plausible answer whose action needs approval pauses for human confirmation. Risky, ambiguous, missing-context or failed-check cases route to the human queue with the full trail and reason codes.

The casebook's approval workflow shows where the AI's own validation sits relative to the human.

Refund Request → AI Validation → Approval Required? ─yes─▶ Manager Approval Queue → Approved → Execute Refund
                                          └──no──▶ Execute

The AI validates first, so most requests never reach a human. Only those crossing a risk or value threshold route to a manager queue before execution. That set is refunds above the threshold, VIP escalations, account deletion, legal or compliance requests, and changes to payment information.

Not every request needs a planner. "Track my order" is a single tool call. "My laptop arrived damaged. Refund the order, cancel the warranty, notify shipping, and create a high-priority ticket" requires planning across Knowledge, CRM, Refund, Ticket, Shipping and Notification agents and merging the results. That is when a planner earns its cost. Simple requests take a single agent or a direct tool call, which is faster and cheaper. RAG (retrieval-augmented generation), tool calling and memory sit side by side as capabilities the orchestrator draws on, not as a pipeline every request passes through. "What is your refund policy?" retrieves from the knowledge base. "Refund my order" invokes a business system. An LLM (large language model) does neither by itself.

Memory is three layers, and keeping them apart is what makes each one fixable.

A stale preference and an outdated policy fail very differently, and conflating them makes both harder to fix. Route models by risk in the same spirit. FAQ traffic goes to a small model, a refund investigation to a large reasoning model. Deterministic rules handle tagging wherever a rule can. Downgrading the model globally harms the few cases where quality matters most; tiered routing, caching, shorter context and better retrieval are the levers.

A thin handoff is operational leakage, not a UX flaw. The agent gets a thinner ticket, repeats the questions the machine already asked, and the customer loses trust. The Handoff record therefore carries the original message, the parsed intent, tool outputs, policy checks, identity status, error codes, the attempted actions and the specific reason automation stopped. The agent sees the triggering intent, the risk rule that fired, the missing facts, the conflicting evidence, or the exact uncertainty that blocked automation. Explainability lives at the workflow level, not the model level.

The human queue sits on the boundary between automation and operations, not deep inside the model path, so high-volume routine traffic never competes with slower manual work. Backpressure lives in three places. Ingress sheds non-essential enrichment on a burst and preserves the conversation. Retrieval degrades to a narrower safe mode when lookups slow rather than waiting indefinitely. Escalation prioritises by risk and customer impact when humans are saturated, never by arrival order alone.

The most revealing interview moment is when security or ops interrupts and injects a bad answer: the assistant confidently claims a refund policy that is wrong. A weak candidate patches the wording. A strong one asks three things first. Was any irreversible action taken? Was a customer misdirected into a harmful path? Can the raw prompt, the model output and the policy version that produced the answer be preserved?

The decision table is the artefact most likely to be requested directly, so hold its six rows in memory.

Replay the path under the failure that matters most: the billing system times out on the double-charge case. The customer sends the message. The gateway checks whether identity is needed. Verification returns an assurance level to the router. The router asks retrieval for policy and account context. Retrieval attempts the billing lookup, and billing times out. The generator still drafts a reply, but the tool policy gateway blocks the action because the source facts are incomplete. The case routes to the human queue with the failure reason and the conversation history. The quality store records the outage and the fallback so routing, escalation and incident review can learn from it. The point of the drill is to make the boundaries visible. Which service can decide, which can only suggest, which state is authoritative, and which failure should trigger a human rather than a retry loop.

Two incident drills follow the same detect, contain, recover, prevent shape. For the wrong policy answer, detection is a policy-answer check or a retrieval mismatch. Containment stops automated delivery, so the answer never becomes a customer promise, and preserves the interaction record. Recovery hands the case to a human with the exact model output and evidence. Prevention adds retrieval grounding, policy versioning, answer validation and a regression test that simulates the same failure. For the refund that times out after possible execution, treat the action as potentially successful and do not blindly retry. Query the ledger or action record to confirm completion, then return success, keep waiting, or escalate. Idempotency keys and ProposedAction.args_hash deduplication prevent the repeat. The answer security wants to hear is not "we will fine-tune it more" but "we will bound it, record it, stop it, and learn from it".

Dependency outages have fixed behaviours. Policy retrieval unavailable: never substitute a stale answer as fresh. Identity provider down: account tools go fully unavailable. Model endpoint failing intermittently: circuit-break, queue eligible tickets, route risky cases straight to humans. Timeouts stay short, retries are bounded and jittered, and dead-letter queues keep failed jobs for inspection instead of dropping them. Stale account state is degraded rather than papered over. Mark the result as potentially outdated, ask the customer to refresh, or route to an agent with the stale snapshot attached. A model that uses old state to make a new commitment is the dangerous case. Language detection that fails never guesses. It falls to a language-neutral intake path with a brief multilingual prompt and escalates when confidence is too low.

Scale from 1 million to 10 million conversations a day with seven moves from the casebook. Stateless gateways, orchestrators and model routing behind load balancers. RAG (retrieval-augmented generation) partitioned by product, region, language and business unit rather than one search over everything. A semantic cache immediately after the gateway, so "What is your return policy?" and "Where is my order?" never reach the orchestrator. Asynchronous workflows for refunds, warranty cancellations and shipping investigations. Rate limits per user, per tenant and per tool, with circuit breakers. Enterprise identity propagated on every tool invocation, with OAuth, RBAC (role-based access control) or ABAC (attribute-based access control) and audit, so the LLM (large language model) never bypasses application authorisation and business systems remain the source of truth. And model routing by task. Regional gateways and orchestrators, indexes partitioned by geography where residency requires, and active-active stateless services complete the global picture.

Deflection measures whether work moved, not whether it was resolved. Safe automation rate, incorrect resolutions and repeat contact measure whether the customer was helped, and each metric is defined by its calculation, source, owner and alert threshold, in one breath.

Measure the right layer for the right question. Technical health asks whether it is alive and safe: latency, tool failure rate, queue depth, circuit-breaker trips, fallback frequency, error-budget burn. Model quality asks whether the AI decides soundly: reviewed-sample accuracy, retrieval precision, escalation correctness, unsupported-answer rate, the share of cases that should have escalated but did not. Adoption asks whether people use it: agent-assist usage, automation acceptance, share of tickets through the new workflow, override frequency. Business outcome asks whether the customer got value: handling time, first-contact resolution, repeat contact, CSAT, cost per resolved case, safe automation rate. Read across layers. Cost per resolved case improving while repeat contact rises means work shifted to a second contact. Safe automation rising while CSAT falls means the wrong intents were automated. Handling time dropping while first-contact resolution stays flat means typing time was saved, not resolution quality.

The go/no-go gate is a checklist, not "the demo looked good":

• retrieval accuracy on the target intent set is stable
• human reviewers approve a representative sample
• fallback behaviour is correct
• logging is complete
• the rollback path has been exercised

Any missing item delays the launch, however fluent the model sounds. Ownership is named. The support operations lead owns adoption and escalation policy. The engineering owner owns reliability and guardrails. Product owns the gate and the scorecard. Security and legal own data access and tool permissions. The frontline manager owns training and day-to-day quality review.

Production trust is earned in layers. The rollout is deliberately boring at first, because each stage proves the next one's precondition.

Each tool expands the blast radius, so each gets its own gate. If one intent drifts, say overconfidence on warranty eligibility, roll back that intent, disable its tool, or drop it to agent-assist while the rest keeps running. Canaries are tiny, visible and reversible, because one bad automated answer multiplies across every similar ticket. Training, a triage playbook for support managers, and documentation of the intent catalog, escalation triggers, approved tool actions and review process are part of delivery. A system the team cannot operate without a developer in the room is not delivered.

The risk register drives the sequence. A hallucinated policy answer is owned by support QA, mitigated by retrieval grounding and mandatory escalation on low-confidence policy intents, and triggered by a spike in incorrect resolutions. A stale account lookup tool is owned by engineering, mitigated by freshness checks and timeout handling, and triggered by mismatches between tool output and CRM records. A segment with unusual wording is owned by operations, mitigated by intent-specific tuning and agent-assist fallback, and triggered by rising override rates.

Separate reusable leverage from customer wiring, because if every customer requires rewriting the same approval logic the result is a consulting script, not a product. Configuration holds intent thresholds, escalation rules, routing weights, language preferences, retention windows and the low-risk intent list. Adapters connect CRM, ticketing, the identity provider, the knowledge base, the survey platform and channels. Shared services are retrieval, policy evaluation, audit logging, prompt assembly, response classification and observability. The core product is the trust boundary, the escalation framework, the safe tool-execution model and the metrics pipeline every deployment needs. The MVP (minimum viable product) is gateway, identity, routing, retrieval, generation, deterministic gating, human escalation and outcome logging. Calibration models, smarter prioritisation, replay pipelines, proactive deflection, multilingual optimisation and automated policy regression tests come later. None of them replaces the control structure.

The same pipeline arrives in two more costumes, and each changes one thing.

The multi-tenant SaaS support assistant makes tenant isolation the dangerous constraint. A support agent handles a tenant-specific question, retrieves that tenant's entitlement, configuration and log context, drafts a response and opens an escalation if needed, never exposing another tenant's data. The delta is a tenant predicate on every retrieval and every tool call, borrowed from the G07 multi-tenant platform anchor. Tenant(id, plan, region, data_residency, features), Entitlement(tenant_id, feature, limits) and RetrievalACL(tenant_id, role, doc_id) gate what the assistant may see. The red-team targets are cross-tenant leakage, wrong entitlement advice, exposed internal logs, and credits issued without approval. The permission-safety metric is zero violations from an ACL (access control list) red-team suite that asks the same question as users from different tenants.

The ServiceNow ticket automation agent is the ITSM (IT service management) flavour. A new ticket arrives. The agent classifies category and priority, retrieves the knowledge base article or runbook, and asks the requester for missing information. It drafts a response, proposes an assignment group, and performs workflow actions only within policy. The records are Ticket(id, requester, category, priority, description, status, assignment_group), Asset(ci_id, owner, service, criticality) from the CMDB (configuration management database), and KBArticle(id, category, validity, approval_status). AssignmentRule(id, condition, group, confidence) and WorkflowAction(id, ticket_id, action, approval_required, executed_by) complete the set. The risks are a wrong priority or assignment, tickets closed incorrectly, unauthorised access changes, leaked requester PII (personally identifiable information) and injection in the ticket description. Mention ITIL change control once. Any action that touches a configuration item goes through the change process the customer already runs, and the agent proposes the change record rather than executing it.

Six study-guide tutorials end with a mock design of this same system in a framework's vocabulary. None changes the design; each adds one implementation detail worth having ready when the interviewer asks "how exactly".

Multilingual ticket triage (LangChain foundations, #95). The pipeline is detect language, classify intent, route, draft and translate back, composed so batching, streaming and retries come free. Routing uses a branch with named intents and a required default arm, so an unrecognised classification never falls through silently. The p95 under 3 s budget splits into an example allocation of about 200 ms language detection, 300 ms classification, 1.5 s draft generation streamed, and 500 ms back-translation, 2.5 s in total. Retries are bounded on every LLM-calling step and the classifier has a fallback provider, because the classifier is the step that must never hard-fail a ticket into limbo.

Human-gated ticket agent (LangGraph fundamentals, #96). Structured-output classification returns {category, risk_level}, and the interrupt lives inside the refund or deletion tool itself, not before the whole turn, so read-only lookups never pause and only the irreversible action does. The checkpointer is durable Postgres keyed on the ticket ID, so a pod restart resumes mid-approval. The refund side effect is keyed on ticket_id + action_type and checked before executing. Order history, account status and prior tickets fan out concurrently and cancel together if one fails. The self-correction loop caps at 3 iterations, then escalates. Below a confidence threshold the ticket goes to human review by default.

Agentic RAG (retrieval-augmented generation) support assistant (RAG and retrieval, #97). Product docs and runbooks are two separate indices, each with BM25 alongside vectors so an exact error code still surfaces. A cheap semantic router decides which to query without an LLM (large language model) call. Retrieve 20 per source concurrently and rerank the merged set to 5 only when the top scores are close. A hard 400 ms budget on the retrieval stage falls back to whichever source returned in time. The generation prompt cites which source each claim came from and says "I don't have that in my docs" below a relevance threshold. A weekly recall@k check on a labelled set built from real tickets gates any retrieval change.

Support search over a fragmented, growing knowledge base (advanced RAG, #99). Three sources, a 5,000-article wiki, ticket history with exact error codes and release notes with version numbers, growing 50 a day, under a 1.5 s p95. Hybrid search per source with α tuned separately, because tickets skew token-heavy and wiki articles skew semantic. CRAG-style grading rather than Self-RAG, because the failure mode is coverage gaps for new error codes, not hallucination, and the below-threshold path logs for a human rather than searching the public web. A cross-encoder reranks the fused top 20, since an LLM judge over 20 candidates would likely eat most of the budget. Each source appends incrementally with a source metadata field, no nightly reindex, and the live fallback rate is the coverage-gap signal.

Travel-booking support agent (agents, #101). A supervisor routes to three specialists: booking_lookup read-only, policy_qa RAG with no side-effect tools, and change_or_cancel as the only agent with write access, so every routing decision is logged in one place. The refund amount is computed as pure logic, not by the model, and over $200 raises an interrupt before the refund tool, whose call is idempotent on booking ID plus request ID. Thread state lives in a durable checkpointer, one thread per conversation; cross-session customer memory lives in a separate store read once at thread start. A hard cap of 15 steps and a per-conversation cost ceiling force a resolution or a handoff. Retry policy applies to the booking API, not to LLM calls; a malformed tool call re-plans through the supervisor. "What's your baggage allowance" never enters the agent at all; it is a direct lookup wired as a workflow.

Support agent over a growing knowledge base (parent-document retrieval, #102). Child chunks of about 400 characters match support phrasing closely; parents of about 2,000 characters bound the worst-case prompt (LangChain's example sizes; tune on your data). Vectors and the docstore share one Postgres, with an HNSW index once exact scans start to eat the latency budget, and the daily 20-article job writes idempotently keyed on hash(article-id + last-modified). The p95 under 2 s budget is a budget of about 100 to 150 ms query embedding, 50 to 100 ms search at 50k vectors (generous; HNSW is often a few ms at that size), 20 to 50 ms docstore fetch, and 1.5 s streamed generation. Cap k at 3 or 4 and return "I don't have a confident answer" below the similarity threshold.

The interviewer's pivot after a good design is "volume is growing and monthly LLM spend is rising faster than ticket deflection". Answer it right after the design, on the same architecture, and reduce cost by workflow rather than across the board.

The sixty-second line: reduce cost by workflow, not across the board. Cache the FAQ path, route by intent, and protect the account-data path. Every strong cost answer is generated by four verbs in order. Measure, by tracing and attributing first. Route, matching model and path to risk. Bound, with limits on steps, tokens, top-k, timeouts and budgets. Cache safely, with tenant, permission and version in the key. Deliver it in six moves: frame the business impact, decompose the path, name the largest measured driver, fix safely, prove with before and after, prevent recurrence. Cost per resolved case that improves while repeat contact rises is the trap to name unprompted.

On this design the levels separate on how the right to act is earned and widened, not on how fluent the drafted reply is.

• The product is a routed decision pipeline that decides when to act, ask, or stop; the harm case, not the architecture, opens the round.
• Requirements are constraints a test can fail: risk-tiered p95 targets, the NetValue equation, and a component owner for every line.
• The automation layer owns workflow state and evidence; CRM, billing, orders and identity remain the systems of record, and every write carries an idempotency key.
• One diagram separates the data plane that carries the conversation from the control plane that decides whether it may continue, with the authorisation boundary at the tool policy gateway.
• Identity is verified before any account tool, and security is a four-seam chain: untrusted text, least-privilege tools, identity, immutable audit.
• A deterministic gateway decides; the model proposes. Planners earn their cost only on multi-system requests, and memory is three separate layers.
• The handoff carries intent, evidence, attempted actions and the reason automation stopped, and the human queue prioritises by risk.
• Failures degrade into a safe handoff by a six-row policy table; the billing timeout and the timed-out refund are the rehearsed drills.
• The release gate is safe automation rate, incorrect resolutions and repeat contact, each with calculation, source, owner and alert, read across four dashboard layers.
• Rollout is agent-assist, then reversible intents, then one tool class at a time, with a kill switch and rollback per intent.
• The tenant variant adds a tenant predicate on every retrieval; the ITSM (IT service management) variant adds classification, assignment and change control.
• The hour is spent on escalation, safety and rollout, with the 90-second summary, the concise close and nine prepared follow-ups.
• Six study-guide mocks supply the implementation detail: branches with default arms, interrupts inside the tool, two indices with a cheap router, CRAG grading, a supervisor with one writing specialist, parent-document sizing.
• The cost pivot is answered by workflow: cache the FAQ path, route by intent, protect account data.

1. Why is "the tool policy gateway is the authorisation boundary" a stronger claim than "we require human approval for refunds"? Because it names where every action is decided, deterministically, on assurance level, allowlist, threshold and freshness, and it makes the model's confidence irrelevant to whether an action may execute.
2. A refund call was issued and the response timed out. What happens next? Treat the action as potentially successful, never blind-retry, query the ledger or action record to confirm completion, then return success, keep waiting, or escalate; args_hash deduplication prevents the repeat.
3. What are the three risk tiers and their p95 targets? Routine under 3 s automated by default; ambiguous under 8 s with retrieval plus a second pass; high-risk under 15 s for a handoff bundle where the final decision is never automated.
4. Why does deflection rate fail as a launch gate? It measures whether work moved, not whether it was resolved; safe automation rate, incorrect resolutions and repeat contact measure whether the customer was helped, and NetValue subtracts wrong-resolution and recontact cost.
5. When does a planner earn its cost? When a request spans systems, such as refund the order, cancel the warranty, notify shipping and create a ticket; "track my order" is a single tool call and must not route through a planner.
6. What must the handoff record contain? The original message, parsed intent, tool outputs, policy checks, identity status, error codes, attempted actions and the specific reason automation stopped.
7. The customer demands 80 percent autonomous resolution in six months. What is the first question? Eighty percent of what; then anchor on intent-specific gates, because the last intents are money-moving and legal where harm cost dominates.
8. What changes for the multi-tenant variant? A tenant predicate on every retrieval and tool call, and a permission-safety gate of zero violations from a red-team suite that asks the same question as users from different tenants.
9. What is the sixty-second cost answer? Reduce cost by workflow: an intent router, a permission-aware FAQ cache, a small model for classification, tools only for account data, and the high-value paths protected.

Fact-check sources (checked 27 Sep 2026)

• Intercom blog: From resolutions to outcomes — vendor-reported 76% Fin resolution rate; deflection range
• LangGraph docs – Interrupts — interrupt inside the tool; durable checkpointer resumes by thread ID
• LangGraph docs – Use the graph API — parallel branches in a superstep succeed or fail together
• Yan et al. 2024, Corrective Retrieval Augmented Generation — CRAG retrieval grading with fallback
• Asai et al. 2023, Self-RAG — Self-RAG trained self-critique
• Sentence-Transformers: Retrieve & Re-Rank — cross-encoder reranking of a small candidate set
• pgvector README — exact search by default; HNSW index option
• LangChain ParentDocumentRetriever source — 400 / 2,000-character example chunk sizes
• Supabase: pgvector v0.5.0 HNSW — HNSW query latency at small scale