Developer vs tester agents

What you'll build

• A developer agent that writes Python and can run it with a timed Python tool.
• A tester agent that reviews the code, writes unit tests in its report, scores it and lists fixes.
• A two-node graph that loops developer, tester, developer until a reflection counter reaches its cap.
• Two small games, FLAMES and Tic-Tac-Toe, built by the pair and checked against hand-written references.

Lesson 1 of T8.2 graded a coding agent on a test suite someone had already written. This project asks a second agent to supply the critique. It is the critique-and-revise (reflection) pattern from T8.1, with a generator (the developer) and a reflector (the tester) taking turns on one shared conversation.

The project adds no new pattern. It shows what two agents in one loop need: a stop rule no model can talk its way past, a safe-enough way to run model-written code, and a line between a test that ran and a test the tester only imagined.

The graph has two nodes and one decision:

Figure 1: The critique-and-revise loop. The router reads a counter in state. Code decides the stop.

Table 1: The project's five pieces.

The notebook checks the Python tool's guards without a model. Keyboard input fails at once with EOFError, and an endless loop is killed at its timeout (2.0 s with a 2-second limit in our run).

The task is Write a Python program for the FLAMES game, plus a short example that calls it with two fixed names, with a cap of one reflection:

Figure 2: One reflection round. The developer speaks twice, the tester once.

The loop's arithmetic holds for any cap. Stub nodes that only append a message, with zero model calls, give these counts:

Table 2: Stub runs from the notebook's own graph builder. The developer runs r + 1 times, the tester r times, and the recursion limit leaves 3 spare steps as a second guard.

After the run, code checks how many times each agent spoke and whether the last answer holds a Python block that parses. Parsing does not prove the game is right.

A hand-written reference does. For John and Jane it finds 4 letters left, crosses out M, L, F, A and S in turn, and lands on Enemy. The notebook records two mistakes one model run made: it restarted counting from the start of FLAMES after each removal, and read S as "Sister" rather than "Sibling". The Tic-Tac-Toe reference plays three scripted games (a win, a draw, a move onto an occupied square) and all three end as expected.

Table 3: Six choices and what each protects against.

• The tester estimates. It has no tools. Its pass percentage and score are the model's estimates, not executed results. We call this an estimated pass rate. Giving it the Python tool makes the results real, at roughly twice the cost.
• Fixed rounds, not "until the tests pass". The loop stops at the cap even if the tester still lists failures. The module's lab brief asks for iteration until tests pass, which needs a router that also reads an executed test result.
• A guard, not a sandbox. The Python tool runs model-written code with your user's permissions. It can still read your files and reach the network, and anything beyond a demo needs a container or a hosted sandbox (lesson 3 of T8.2).
• No recorded model run. The notebook ships without outputs and the drafts change each run. This page quotes only the deterministic checks.

The notebook needs an OpenAI key for the two agents. The stub runs, the tool guards and both references run without one.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• Stop a two-agent loop with a counter in state and a pure router, back it with a recursion_limit, and prove the counts with stub nodes before spending a model call.
• A tester without a way to run code only estimates its pass rate. Run the tests, or check against a reference, before trusting the verdict.
• Running model-written code in a fresh process with a timeout, no environment and closed input keeps a demo from hanging or leaking keys, but it is not a sandbox.