Explainable retrieval

What you'll learn

• Trace an explainable retriever: one search, then one explanation call per chunk.
• Diagnose an explanation that rationalises a ranking error.
• Separate explaining a chunk to a user from debugging a ranking.
• Redesign the explainer as a relevance grader.

Lesson 2 of T6.4 linked each answer to the chunks that support it. A user can still ask the step before that: why did the system pick these chunks at all?

Notebook 03a answers with an explainable retriever. It wraps a FAISS index with k = 5 and, for every chunk it returns, makes one more LLM call that writes a paragraph on why the chunk fits the query. The output is a list of pairs: the chunk's content and its explanation.

Figure 1: One retrieval, then one explanation call per chunk.

Three properties of this design decide where it is useful.

The loop calls the explanation chain k times, one after another. At k = 5 a single search becomes five model calls before any answer is written:

Table 1: Calls per question for three explanation policies.

Run the calls as one batch to cut the wait, or explain only what a reader will look at.

The prompt says "Explain why this context is relevant to the query". The model never sees the similarity score or the other chunks, and it is not offered "this chunk is not relevant" as an answer.

On the Larkspur handbook that matters. In the T5.1 TF-IDF index, What happens if my server goes down? ranks the reboot chunk first, on the shared word "server" (lesson 2 of T5.1). Asked why reboot is relevant, with no way to say it is not, the model's likely reply is a reason:

Table 2: An explanation for a ranking error (illustrative reply).

That reads as an explanation and is a rationalisation of a ranking error. We call it the rationalised miss. The chunk was ranked by similarity between two vectors; the paragraph is the model's reading of the query and the chunk side by side, written after the fact.

Key distinction: an explanation can guide a user and mislead an engineer. To debug retrieval, print the scores and the rank of the chunk you expected, never the explanation.

A version that helps both readers changes the question:

Table 3: From explainer to relevance grader.

The yes-or-no form is a relevance grader. On the reboot chunk it can return "no", with the score beside it, which is the signal the explainer could never give. T9.1 uses it to filter chunks and rewrite the query, and lesson 1 of T9.5 adds a hallucination check and source highlighting on top.

Two notes on running 03a today. Its retriever call uses a method that the current LangChain core no longer has, so it raises an attribute error; the current call is the retriever's standard invoke. And the demo indexes three sentences with k = 5; FAISS returns all three, so each query gets three explanations, including two for texts about photosynthesis and global warming when the question is why the sky is blue. That is forced top-k (lesson 1 of T6.1) again, now with a paragraph defending each forced pick.

1. Explain on demand: a support agent clicks a source to see why it is there; no call is spent otherwise.
2. Grade, then explain: ask the yes-or-no question first, and show the verbatim evidence quote as the explanation.
3. Show the score beside every explanation, so a weak match is visible as weak.
4. Debug with ranks, logged per query, never with explanation text.

Lesson 4 of T6.4 leaves free text: Larkspur's server catalogue is a table, and a table row needs its own representation before it can be retrieved.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• An explainable retriever adds one LLM call per chunk. Batch the calls, or explain only the chunks the answer cites or a user expands.
• A prompt that asks why a chunk is relevant gets a reason even for a ranking error (the rationalised miss). The paragraph explains the text, not the ranking, so debug retrieval with scores and ranks.
• Ask a yes-or-no relevance question with a verbatim quote and the retriever's score instead. That turns the explainer into a relevance grader that can say no.