Adaptive RAG & Query Routing: overview

A fixed RAG pipeline does the same work for every question. A support desk cannot afford that: one Larkspur ticket needs billing policy, another a technical guide, a third a person, and a fourth something the handbook does not hold. A router reads the question first and decides where it goes, and every step after it can only work with what that route returns. This module covers how a router classifies a question into a closed set of labels, how a label becomes a knowledge-base filter, when a ticket should go to a person instead, how to choose between the vector store and the web and then check the evidence and the answer, and how to pick a retrieval method per kind of question.

The running example is the Larkspur Cloud support desk. The twelve handbook chunks from T5.1 are split into the three knowledge bases the 01a notebook routes between: billing (five chunks), technical (four) and general (three). The lab routes eighteen support tickets, each labelled with the knowledge base that holds its answer, such as Why was I billed twice this month? (billing) and How long do you keep my data after I cancel? (general).

• A router is a classifier in front of retrieval. In the notebooks it is one LLM call with structured output that must return a label from a fixed set (Literal['Technical', 'Billing', 'General']), and the label becomes a metadata filter or a branch in a LangGraph graph.
• A misroute hides the answer: the filter restricts retrieval to a knowledge base that does not hold it. In the lab, a router that always picks sends 15 of 18 tickets to the right knowledge base at one search each. A score floor of 0.42 removes all 3 misroutes, but 11 tickets then fall back to searching all three knowledge bases, 2.22 searches per ticket.
• Escalation is checked before the category. A negative ticket goes to a person whatever it is about; the healthcare notebook adds a fourth sentiment, Distress, that goes to an on-call team.
• Adaptive RAG routes between the vector store and the web, then checks twice: a document grader sends the question to the web when half or fewer chunks pass, and a hallucination grader sends an unsupported answer back for regeneration.
• A strategy per query type spends LLM calls where the question needs them. In the 04a notebook a factual or contextual question costs up to 11 LLM calls and an analytical or opinion question 4.

The router decides what retrieval is allowed to see. Sentiment is checked first, so an unhappy customer reaches a person whatever the topic; the rest go by category to one knowledge base, which retrieves, and the model answers from what came back. Grading and generation work only with what the chosen path returns, so a misroute surfaces as a confident wrong answer or an apology, never as an error. Lesson 1 of T9.3 calls that a silent misroute, and every lesson in the module has a version of it.

Routing confidence and the fallback: open the file in a browser. Start at floor 0.00 and find the three misroutes in the ledger. Move to 0.01: the ticket that shares no word with any chunk falls back, and the cost rises only to 1.11 searches per ticket. Then push the floor to 0.42 to catch the last misroute, and read what it costs: 11 of 18 tickets now search all three knowledge bases.

• Before: T6.2 introduced logical and semantic routing and metadata filters built from the question; this module puts a router at the front of a whole graph. T4.4 covers checkpointers and interrupts, which a production escalation path needs. T9.1 and T9.2 built the document grader and the hallucination check that lesson 3 of T9.3 combines with routing. The lab reuses the handbook and the TF-IDF index of T5.1.
• Next: T9.4 hands retrieval to an agent as one tool among many, so the agent's choice of tool becomes the router. T9.5 covers source attribution, unsupported-answer detection and feedback loops for systems like these.

• Give the router a closed label set in its output schema, and normalise the label before it becomes a filter.
• Plan for the silent misroute. A score floor with an unfiltered fallback catches no-signal routes cheaply; confident misroutes only show up on a labelled routing set.
• Check escalation before the category, and make the handoff a pause with saved state rather than a node that blocks while it waits.
• Every grader loop needs a counter, and every query type should spend only the LLM calls its question needs.