Dry-run harness and human approval

What you'll learn

• Explain what a dry run is and the three properties that make its preview trustworthy.
• Trace a paused graph from interrupt() to Command(resume=...) on the same thread.
• Diagnose an unclear answer and an edit made after review, and say why each one must stop the post.
• Choose where the dry run sits in the graph, and what the approval has to cover.

T8.2 gave agents real reach: they write files, run commands and drive a browser. Every one of those agents ends a turn by doing something, and so far nothing stood between the model's decision and its effect.

Larkspur's operations agent drafts posts for the status feed. After the eu-west outage it writes "The eu-west outage is resolved and all servers are back online. Engineers will publish a root-cause report within 72 hours of recovery." With a plain tool call, the model proposes the post and the same call publishes it. A wrong number, a promise the engineers never made or a jab at a competitor reaches every customer before anyone reads it.

A plain tool call joins two moments that need to stay apart: the model proposing an action and the system executing it. This lesson separates them with two pieces. A dry run shows the reviewer the exact effect. An approval gate pauses the run until a person decides, and ties that decision to the preview they saw.

A dry run executes an action's plan without its effect, and reports what the effect would be. The idea is old: rsync --dry-run, apt-get --simulate and terraform plan all print the changes before anyone applies them.

The lesson's tool is a mock social-media API that stands in for the Larkspur status feed. It has one method that publishes a post, with a dry-run switch, and one rendering step shared by both modes:

Figure 1: One code path, two outcomes. The preview and the published text come from the same rendering step.

The two calls return different things, and only one of them touches the feed:

Table 1: The two modes for the eu-west post. The fingerprint is the lab's value.

The fingerprint is the first 12 hex digits of a SHA-256 hash over the rendered text, post body plus hashtags. It names the exact text that was previewed, and the approval gate relies on it.

Three properties that make a preview trustworthy

A preview is worth reading only if it predicts the live call. Three properties make that hold:

• The preview comes from the live code path. Both modes render the post the same way. The reviewer reads the text the platform will receive. A preview built by a separate "describe the action" function can drift away from what the live call does. Call that a drifting preview: the reviewer approves one thing and another thing runs.
• "Nothing happened" is checked, not assumed. The mock feed is a list the notebook can read back, and every claim that a dry run published nothing is an assertion that the feed is still empty. A real tool needs the same: a sandbox, a staging target or a way to read state back.
• The dry run is its own graph node, before the pause. On resume, LangGraph re-runs the paused node from the top. Anything placed before interrupt() inside the review node runs a second time. A preview generated there could differ from the one the reviewer approved.

The third property hides until the graph resumes:

Figure 2: What a resume does. Only code that is safe to repeat may sit above the pause.

The cost: every consequential tool needs a preview

Each tool that changes the world needs its own preview, showing what the reviewer must judge:

Table 2: Preview contents per action (illustrative examples beyond the notebook's post).

With a trustworthy preview in hand, the next job is to stop the run until someone has read it.

An approval gate is a node that pauses the run, hands the preview to a reviewer and resumes only with their decision. In the harness the gate is the review node. It calls interrupt() with three things: the question, the preview and the fingerprint.

Parking and resuming the same thread

LangGraph stops the run at that call, saves its state in the checkpointer under the run's thread_id, and returns the payload to the caller under __interrupt__. Nothing more happens until someone invokes the graph again with Command(resume=decision) on the same thread:

Figure 3: One pause and one resume. The state waits in the checkpointer for as long as the reviewer takes.

The notebook wraps that exchange in a small pause-and-resume loop: invoke, and while the result holds an interrupt, pass the payload to a reviewer function and resume with its answer. Only the reviewer function changes between uses:

• A script: approves the launch post and rejects the post that says the new model makes every competitor obsolete. An automated run then shows both outcomes.
• A policy check: an automated rule, configured explicitly.
• A console prompt: a person types the answer.
• A review UI: answers hours later, from another process.

An answer that arrives hours later from another process needs a persistent checkpointer, such as SQLite or Postgres, in place of the notebook's in-memory one.

Binding the decision to the preview

Pausing is the mechanism. Three rules make the gate safe:

• Fail closed. The decision is normalised by stripping spaces and lowercasing, and only the exact word approve approves. Everything else counts as a rejection.
• Approve what was previewed. The approval stores the preview's fingerprint. The publish node recomputes the fingerprint of the post it is about to send and refuses if the two differ.
• Log every decision with the reviewer, the reason, the fingerprint and the time.

Here is the fail-closed rule against the answers reviewers actually type:

Table 3: Planted answers through the decision rule. An ambiguous answer never publishes.

A console, a script and a web form all feed the same rule, and the audit entry keeps the original answer beside the decision. The opposite design, where "yes" counts, is a lenient gate. It approves whatever a person half-meant.

Planted failure: an edit after approval

The second rule is the one that catches post-approval drift, any change between what the reviewer read and what is about to run. The notebook plants it by editing the paused post directly in the checkpointed state, appending " Buy now!!!", and then approving.

The approval-gate lab runs the same harness in the page, with the decision rule and the SHA-256 fingerprint ported from the reference implementation. Checked against the reference graph on 315 runs (3 drafts, 3 kinds of edit, 35 typed answers), it agrees on every final status, fingerprint and audit entry. With the eu-west post approved:

Table 4: One approval, three versions of the post. The fingerprints were recomputed from the lab's post text.

A one-phrase edit changes the fingerprint as completely as a new sentence. Any edit after review needs a new preview and a new decision. In both refused runs the audit log still shows an approve. That entry is correct. The reviewer approved c4e9e330698e, and the text that tried to publish was something else.

Pitfall: the gate is only as good as its reviewer. Approve the uptime post in the lab ("Larkspur Cloud now beats every other host on uptime") and it reaches the feed. The gate enforces the decision, and it cannot judge the content. The module page's interview questions cover how to measure whether reviewers read.

The same pause exists ready-made. HumanInTheLoopMiddleware from T7.2's middleware lesson gives a create_agent agent a pause before a tool call, built on the same interrupt() and checkpointer from T4.4. Building the gate by hand adds the two parts this lesson needs: a preview of the effect, and a fingerprint that ties the approval to it.

Here is the whole harness on Larkspur's post, in the order the graph runs it:

1. Propose. The model drafts the post and its hashtags as structured output.
2. Dry run. Its own node renders the text and computes c4e9e330698e. The feed is still empty.
3. Review. The graph pauses and the checkpointer holds the run. The on-call lead reads the preview.
4. Decide. The lead types approve. The audit log records the reviewer, the reason, the fingerprint and the time.
5. Publish. The recomputed fingerprint matches, and the post goes out as post_1. An edit in the meantime would have been refused.

Lesson 2 of T8.3 builds agents that only read: they search the web and return a brief. Their risk is a wrong claim, and the guard moves from a person's approval to checks in code.

The harness tested an unclear answer and an edit after the preview, on a mock feed. It leaves out:

• Staleness. The fingerprint covers the text, not the world it describes. A post approved four hours later still matches even if the incident reopened. Add an approval expiry or a precondition checked at publish time.
• Persistence. The in-memory checkpointer loses a paused run on restart. Add SQLite or Postgres before reviewers answer from another process.
• Reviewer quality. Nothing measures whether approvals are read. Add approval-rate and time-to-decision monitoring.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• A dry run renders the effect through the same code path as the live call and changes nothing. Check that by reading state back, and put the dry run in its own node before the pause.
• The gate pauses with interrupt(), saves state under the thread_id and waits for Command(resume=...). Accept only the exact word approve and log every decision with reviewer, reason, fingerprint and time.
• Tie the approval to a fingerprint of the previewed text and recompute it before publishing. Any edit after review, even one phrase, needs a new preview and a new decision.