Every correction is another LLM call.
The twelve Larkspur Cloud handbook chunks from M10, five questions, and the routing code of the M29 notebooks. Pick a question and move k: the ledger counts the calls each pipeline makes and shows what its generator finally reads.
- LLM calls, CRAG
- LLM calls, Self-RAG
- CRAG searched the web
- Self-RAG retrieval passes
Three pipelines on this question
| Pipeline | LLM calls | Route, and what the generator reads |
|---|
Magenta chunk ids are the question’s evidence. The graders here are stand-ins with fixed rules: relevance agrees with the golden labels, groundedness says yes, utility says yes only when every evidence chunk reached the generator, and a rewrite repeats the question, so retrieval never improves. That makes the Self-RAG counts a worst case.
All five questions at this k
| Question | Baseline | CRAG | Self-RAG |
|---|---|---|---|
| Mean LLM calls |
The retriever returns up to k chunks with a TF-IDF score above zero, like the score-threshold retriever in notebook 01a. CRAG follows 01a: grade each chunk, and rewrite and search the web when half or fewer are relevant. Self-RAG follows 02a: decide to retrieve, grade, generate, check groundedness and utility, with max_iterations = 2. A web search is a tool call, not counted as an LLM call.