Evaluator–optimiser loops, always bounded: LangGraph vs LCEL

What you'll learn

• Explain how an evaluator-optimizer loop feeds a critique back into the next draft.
• Design criteria an evaluator can pass, with the verdict computed in code.
• Prove that a loop ends when the evaluator never approves, and count its worst-case calls.
• Choose where the loop lives: LangGraph, a Python loop around LCEL, or agent middleware.

Lessons 1 to 4 of T7.3 all run forward: every step happens once, or a fixed number of times. The 24.1 gate could retry, but it only re-ran extraction blind. This lesson's pattern sends the reason for a rejection back to the step that can fix it.

A Larkspur answer can cite the handbook and still promise a refund the policy does not allow. A reviewer that names that problem, and a generator that revises with the review in hand, can catch it. The risk is a reviewer that never says yes.

Generate, evaluate, revise

Evaluator-optimizer generates a draft, grades it against explicit criteria and feeds the critique into the next draft, until the grade passes or a cap is reached. The 24.1 gate decided whether a result may proceed; this loop sends feedback to the generator. It pays off when two things hold: the criteria are clear, and feedback measurably improves the next attempt.

The lesson's loop writes a short joke about agentic AI:

Figure 1: The bounded loop. Approval ends it, and so does the fifth draft.

The generator runs at temperature 0.7, so a revision can differ from the draft before it. The evaluator runs at 0 to keep grading as consistent as possible, though the same draft can still get a different grade on another call. From the second pass on, the generator receives the previous draft and the feedback, so it revises instead of starting over.

Criteria an evaluator can pass

The evaluator is only as useful as its criteria. A first version of the notebook asked for one holistic grade against loose criteria such as "is it genuinely funny?", and with gpt-4o-mini the loop almost never converged: the evaluator kept finding something that could be better. We call this the moving bar. The bounded version changes three things:

Table 1: From a holistic grade to computed criteria.

Proving the loop stops

A structured verdict does not guarantee the evaluator will ever approve. So the router checks two things in order: approval first, then the iteration count against a cap of 5. We ran the reference stopping rule on four states:

Table 2: The router's decisions. Approval is checked first, and no state starts a sixth draft.

The notebook proves the cap on a live generator with a stub reviewer that rejects every draft and makes no LLM call. Without the cap branch, that cell would never finish. With it, the run ends after five drafts.

The cap also bounds cost. Each pass is one generator call and one evaluator call:

Table 3: Calls per run under a cap of 5.

Planted failures

Each guard has a case that shows why it is there:

Table 4: One planted case per guard.

Pitfall: the capped success. A run that ends at the cap returns a draft that did not pass. Count it as a failure, and watch the share of runs that end there: it rises when criteria drift or a new evaluator model is stricter than the old one.

Where the bound lives

The bound sits in a different place in each implementation:

Table 5: Three homes for the same bound.

For Larkspur, an answer-quality check is the natural evaluator, written as yes-or-no criteria: does the reply cite a handbook chunk, does it state the refund window the chunk states, does it avoid promising anything the handbook does not say, and is it under a length limit counted in code. CRAG (T9.2) applies the same loop to retrieval: grade the documents, rewrite the query, try again, within a bound.

LCEL, the LangChain Expression Language, composes runnable steps into a flow. Every earlier T7.3 pattern is a flow without cycles, so each maps onto an LCEL building block. This one has a cycle, and LCEL has no block for "go back and try again until a check passes":

Table 6: The five patterns and their LCEL coverage.

Two tempting shortcuts fail:

• Piping generator, evaluator, generator builds a three-step chain that runs the generator twice. It never loops.
• A retry wrapper re-runs a step blind, with no memory of why the last attempt failed. Feeding the critique forward is the point of the pattern.

Choosing the runtime around the loop

The two LangChain loops differ in what surrounds them:

Table 7: Two ways to loop without LangGraph.

In the middleware version the hook must declare that it may jump back to the model; that declaration is what creates the conditional edge. When the loop needs checkpoints, interrupts or a human in it, move it to LangGraph.

• No recorded convergence counts. The notebooks ship without outputs, so how many passes a real joke needed is not measured here; the "almost never converged" result is the notebook's own report on its first version.
• Model-free router trace. Table 2 runs the reference stopping rule on hand-set states, not a live loop.
• One task. Jokes are a cheap stand-in; Larkspur criteria were not run.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• Evaluator-optimizer feeds a critique into the next draft until the grade passes. Use it only when the criteria are clear and feedback measurably improves the next attempt.
• Grade with one yes-or-no per criterion, check length in code and compute the verdict from the checks. A single holistic grade becomes a moving bar the loop rarely clears.
• Cap every loop in your own code, count a run that hits the cap as a failure, and do not rely on the framework's default limit. LCEL cannot express the cycle, so use a Python loop or middleware, and LangGraph when you need checkpoints or interrupts.