RAG as an Agent Tool: overview

Larkspur needs handbook evidence for a policy answer, but small talk should not trigger a search. This module lets the agent choose when to call a retriever tool, measures the description that drives that choice, and then moves the retriever behind a managed service and the agent behind a chat window.

The running example is the Larkspur Cloud support agent. It has two tools: handbook_search, a retriever over the handbook from T5.1 to T5.3, and web_search for everything outside Larkspur. The lab routes twenty mixed questions between them, ten about the handbook (How long does a refund take?, Can I export my data after I cancel my plan?), six for the web (What does Hetzner charge for a 4 GB server?) and four that need no tool at all (What is 15% of 240?).

• As a tool, retrieval runs inside a loop: the agent node calls the model, the model may reply with a tool call, ToolNode runs the retriever, and the chunks go back to the model as a ToolMessage. The loop ends when the model replies without a tool call. The default retriever tool drops each chunk's source, so print it into the chunk text when the answer must cite.
• All the model knows about a tool is its name, its description and its argument schema. In the lab, where TF-IDF stands in for the model, two tools both described as "Search tool." route 4 of 20 questions correctly, and only the four that needed no tool; specific descriptions route 15 of 20.
• Tune descriptions like any retrieval setting: on a golden set, then on questions held out from it. Fitted to the lab's twenty questions they route 19 of 20 and 4 of 5 held-out questions, and they cost 133 tokens on every model call instead of 84.
• A managed RAG platform (Contextual AI in lesson 2 of T9.4) runs the datastore, parser, reranker and grounded generator behind one API. The agent can call the whole platform as a single tool, so swapping it in leaves the agent loop unchanged.
• A chat UI adds sessions, settings and visible tool steps. State such as the index and the agent belongs to the user's session, not to module globals.

Retrieval sits inside the loop, so the model decides whether it runs, how often, and with which query. A question that never triggers a tool call is answered from the model's weights, with no handbook text in front of it. The quality of the retriever matters only for questions the model sends to it.

Tool descriptions and routing: open the file in a browser. Start on Names only and see all sixteen handbook and web questions get no tool. Switch to Specific and find the one question sent to the wrong tool, then add the word "refund" to the handbook_search description and watch How long does a refund take? move and the score reach 16 of 20. Finish on Tuned and compare its golden-set score with its held-out score.

• Before: T7.1 and T7.2 teach tool calling and the ReAct loop this module puts a retriever into. T9.1 to T9.3 make the retrieval decision inside a fixed graph; here the model makes it. T5.1 gives the retriever and the golden-set habit the lab reuses.
• Next: T9.5 checks whether an answer is supported by what was retrieved. T10.1 adds conversation memory to the chat. T12.3 exposes retrieval as a tool over MCP, so any agent can call it.

• An agent retrieves only when the model calls the tool. Measure how often it does on questions that need the handbook, and how often it does not.
• Write each tool description for the model: what it searches, the topics in users' words, and where it stops and the neighbouring tool starts. Keep it short, because it is sent on every call.
• Tune descriptions against a routing golden set and score them on held-out questions too, as you would any retrieval setting.
• A managed platform or a chat UI changes what sits behind or in front of the tool, not the loop. Keep the tool's name, description and arguments stable across both.