Threat Model Each Request Path

A list of AI threats tells you what could go wrong somewhere. A threat model tells you what goes wrong on this path, through this door, with this much authority.

Generic threat lists ("prompt injection, data leakage, jailbreaks") are a starting vocabulary, not a design. A useful threat model follows one request from the moment it enters the system to the moment something changes in the world, and asks the same four questions at every hop. This lesson gives you those questions and a way to write the answers down so that each threat ends in a named control.

Objectives:

• trace a request path and mark every point where untrusted content enters;
• describe a threat by attacker, asset, entry point and authority boundary;
• turn each threat into a control that lives outside the model;
• run the exercise in an interview without drifting into a generic checklist.

Draw the path first. For an agent that answers support tickets and can issue refunds, the path is: the ticket arrives, the agent retrieves policy documents and the customer's order history, the model plans, it calls the refund tool, and the payment system moves money. Five hops, three data sources, one action that cannot be undone.

Now mark every point where content the system did not write enters the model's context. Here there are three: the ticket text, the retrieved documents and the tool results. Each is an entry point. Treat everything that arrives through one as data that may contain instructions. This is the same rule Prompt Injection and Egress Control applies to a single agent, extended to the whole path.

Then mark where authority changes. The ticket author has none. The agent has whatever its tools allow. The payment system trusts the refund tool completely. Each change is an authority boundary, and boundaries are where attacks pay off.

Write every threat as one row with four parts.

A threat written this way is specific enough to test. "Prompt injection" is not. "A customer writes text into a ticket that makes the agent refund a different order to a new account, using the agent's payment authority" is.

Most serious agent attacks borrow authority rather than steal it. The attacker has little power of their own. The agent has a lot. The attack succeeds when the agent spends its authority on the attacker's behalf.

So for each row, ask: whose authority executes the final step? If the answer is "the agent's service account" and that account can act for any customer, the attack is a confused deputy. The fix is to shrink what the agent can do for this request, which the next lesson, Delegated Authority and Zero Ambient Authority, covers in full.

Every row needs a control, and the control must hold even if the model is fully persuaded by the attacker. Assume the model will do whatever the injected text asks, then ask what still stops the harm.

A control that says "the system prompt tells the model to ignore instructions in tickets" is not on this list. It may reduce how often the attack works. It does not bound what happens when it does.

You will find more rows than you can fix in one release. Rank them by two things: how bad the outcome is, and whether it can be undone. A wrong answer in a chat can be corrected. A refund to the wrong account, a deleted record or a message sent to a customer cannot. Irreversible actions with external effect go to the top, whatever their likelihood.

Write down the rows you accept for now, who accepted them and when they will be reviewed. An unranked list of fifty threats is as useless as no list. A ranked list with three owned exceptions is a release decision.

A research agent reads public web pages, summarises them and writes the summaries into the company's internal knowledge base, which other agents later retrieve from.

The path has two entry points worth a row each. Web pages enter during research. The knowledge base enters later, when a different agent retrieves what this one wrote.

• Row 1. Attacker: anyone who can publish a web page. Asset: the internal knowledge base. Entry point: page text. Authority boundary: the agent's write permission on the knowledge base. Control: summaries are written to a review queue with the source URL attached, and only reviewed entries become retrievable.
• Row 2. Attacker: the same. Asset: the behaviour of every agent that retrieves later. Entry point: a stored summary that carries instructions. Authority boundary: the later agent's tools. Control: retrieved knowledge-base text is treated as data in every consuming agent, the same as web text.

Row 2 is the one teams miss. The write-back turned a one-off injection into a stored one that fires in a different system, days later. The lesson on Tool Poisoning, Memory Poisoning and the Supply Chain covers this class of attack.

• Start from one request path and mark where untrusted content enters and where authority changes.
• Write each threat as attacker, asset, entry point and authority boundary, specific enough to test.
• Most agent attacks borrow the agent's authority, so ask whose authority executes the final step.
• Every threat needs a control that holds even when the model is fully persuaded.
• Rank by impact and reversibility, and record the threats you accept with an owner and a review date.