Reflection and Reflexion agents

What you'll learn

• Explain the reflection loop: one model in two roles, with the conversation as the state.
• Diagnose three ways self-critique fails: a rule nobody checks, an invented fact and a critic that believes its own guess.
• Compare reflection with Reflexion, which turns the critique into searches and cites what it finds.
• Choose the critic for a task by asking what evidence it can reach.

Lessons 1 and 2 of T8.1 checked whether a search result answered its step. This lesson turns the check on the model's own output: draft, critique, revise.

Larkspur's status team wants an agent that drafts a short post after an incident, under 180 words. A reflection loop sounds ideal: a writer drafts, a critic says what is weak, the writer revises. In practice the critic asks for "a statistic to build trust", the writer adds "99.99% uptime", and the post grows past its limit. The handbook guarantees 99.95%, and nothing in the loop ever looked.

A reflection agent is one model playing two roles in a loop. The generator drafts and revises. The critic (or reflector) reviews the latest draft and returns free-text, actionable feedback. The idea comes from Self-Refine (Madaan et al., 2023). The reflection notebook drafts LinkedIn posts. The mechanism is the same for Larkspur's status post:

Figure 1: The reflection loop with the notebook's cap of three drafts, the first draft plus two revisions.

In LangGraph the whole exchange is one message list, and two details make it work. The generator sees each critique as the next human message and answers it with a revised draft. The critic sees the conversation with the roles swapped, which makes the writer's drafts arrive as the user's input it has to review. A pure router counts the drafts written and stops at MAX_DRAFTS = 3.

The finished state holds six messages:

Table 1: The message trace of one reflection run. The router counts the AI messages.

Two LLM calls per round, plus the first draft, gives 5 calls for three drafts. The cap is a count in code. In an agentic version the critic could also decide when the draft is good enough, which needs a cap even more, because a critic can be too easily satisfied or never satisfied.

The loop gives the model more attempts. It does not give the critic anything to check against. Three recorded failures show what that costs:

Table 2: Three failures of self-critique, from the reflection notebooks' recorded runs.

Each one gets a name we reuse. The unchecked rule is a constraint in the brief that no step verifies. Revisions drift past it. The invented fact is a number or claim the writer supplies because the critic asked for one. Self-bias is the critic flagging only the problems it can imagine and trusting its own guess about the rest.

The fix for the unchecked rule is to check it in code. Word counts, required phrases and formats are deterministic, which is why T7.3's evaluator-optimizer moves checkable criteria out of the critic. The other two failures need evidence, which the next two sections add in different forms.

The optional reflection-architecture notebook runs the smallest structured version: generate, critique, refine, once, with no loop. Each role returns a typed object. The critique has fields has_errors, is_efficient, a list of suggested improvements and a summary. Its verdict becomes a claim we can test.

The task is a duration parser that turns strings like "1h30m" or "2h5m10s" into seconds. The request says nothing about bad input. Ten hidden tests, which neither generator nor critic sees, define what a reasonable user expects:

Table 3: The ten hidden tests, run in code against the draft and the refined version.

The notebook runs the whole pass four times and compares the critic's belief with the measured score. Refinement helps when the critique happens to name what the tests check ("validate the input", "raise a ValueError for unknown units"). It is not reliable. Across runs the refined code can pass the same number of tests as the draft, or fewer, when a refiner "simplifies" the parser with a regular expression and drops validation. One run is an anecdote, which is why the comparison repeats.

Pitfall: self-bias. A critic that cannot run the code, look up the fact or see the rule's check will sound equally confident when it is wrong. Treat its verdict as a hypothesis and measure the output.

Reflexion (Shinn et al., 2023) extends reflection in two ways: the critique must say what is missing, and the gaps turn into external research that the revision has to cite. In the paper an actor attempts the task, an evaluator scores it, and a self-reflection step writes a verbal lesson that is kept in memory for the next attempt. The paper reports a Reflexion agent reaching 91% pass@1 on the HumanEval coding benchmark against 80% for GPT-4 without it, and 130 of 134 AlfWorld tasks solved.

The Reflexion notebook implements the within-task form with a search tool:

Figure 2: The Reflexion loop. The model's own queries are what the search step runs.

Three design decisions carry the pattern:

• One structured response. The draft holds the answer, a reflection with two fields (what is missing, what is superfluous) and 1 to 3 research queries. The schema is bound as a tool the model must call, and the response arrives as a tool call that a tool node can "execute" by running its queries.
• Strict schemas, with a retry behind them. Without OpenAI's strict schema mode, gpt-4o-mini left out the revision's required sources field in 3 of 4 test calls, often on every retry. A responder that feeds the validation error back and asks again (up to 3 attempts) stays as the fallback for providers without strict mode.
• A revisor that must change something. With the first guidelines the revision copied the draft almost word for word, attached sources without citing them and re-issued the same queries. Two added rules (add at least one fact from the results, cite it as a numbered reference) and a temperature of 0.7 fixed that, because at temperature 0 a second revision repeated the first.

The cost of one run follows from the graph:

Table 4: One Reflexion run with MAX_REVISIONS = 2.

The router counts revisions directly. The notebook's first version counted trailing messages, which hid the real limit. After the run, code checks the guidelines itself: a body of at most 300 words, numbered citations in the text and at least one source URL. The model's claim to have followed them is not evidence.

Key distinction: reflection asks "what would make this better?" and answers from memory. Reflexion asks "what is missing?" and goes to get it.

The critic is only as good as what it can reach. Four kinds appear in this module, and each catches a different failure:

Figure 3: Pick the strongest critic the task allows, and fall back to plain reflection only for judgement calls.

Table 5: The critics side by side.

For the status team's post, the choices combine:

1. Draft and critique with a reflection loop for tone and clarity, capped at three drafts.
2. Check the limit in code: the word count and the required link to the status page.
3. Ground every number Reflexion-style, with the handbook as the search tool. The critique names "uptime figure unverified", the query retrieves the SLA chunk, and the revision cites 99.95% monthly availability and the 72-hour root-cause report.
4. Score in code after the loop, and route a post that still fails to a person.

Lesson 4 of T8.1 makes the fact check stricter by answering one question per claim in a call that never sees the draft.

1. Madaan et al., Self-Refine: Iterative Refinement with Self-Feedback, NeurIPS 2023.
2. Shinn et al., Reflexion: Language Agents with Verbal Reinforcement Learning, 2023.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• Reflection is one model as writer and critic, with the conversation as the state and a cap on drafts. It improves what the critic can judge from opinion, mostly style and structure.
• Self-critique fails on unchecked rules, invented facts and self-bias. Check hard rules in code, and measure the output instead of trusting the critic's verdict.
• Reflexion makes the critique name what is missing and turns it into searches that the revision must cite, at 3 LLM calls and up to 6 searches for two revisions.