RAG-Fusion with Reciprocal Rank Fusion

What you'll learn

• Explain what Reciprocal Rank Fusion keeps that a flat union throws away.
• Trace the Larkspur outage question through five variants and a fused list.
• Predict when the constant k changes the order, and compute both orders by hand.
• Diagnose agreement on a distractor, the failure fusion cannot see.

Lesson 2 of T6.2 merged three phrasings with a flat union and found the evidence for the outage question, then ranked it 5th by order of arrival. The union keeps every chunk once and throws away two signals: where each list ranked the chunk, and how many lists found it.

RAG-Fusion keeps both. It asks for five phrasings, because agreement means little across two or three rankings, and fuses the lists with Reciprocal Rank Fusion (RRF). Notebook 03a scores each chunk by adding 1 / (rank + k) for every variant that retrieved it, with rank counted from 0 and k = 60:

Figure 1: RAG-Fusion. The fusion step needs only ranks, a dictionary and a sort.

With k = 60, first place in one list adds 1/60, about 0.0167, and fourth place adds 1/63, about 0.0159. Being found by more variants therefore matters more than placing first in any one of them. RRF uses ranks and never similarity scores, because scores from different queries are not on the same scale.

T6.1's fusion page covers the same formula with the more common convention that counts ranks from 1. There the lists come from BM25 and dense retrieval; here they come from five phrasings of one question.

The interactive query-variant lab runs RAG-Fusion on the T5.1 index: the same twelve Larkspur chunks, the same TF-IDF, and five hand-written variants per golden question in the style of the 03a prompt. Each variant retrieves its top 4 (LangChain's default k), minus chunks it shares no term with. For What happens if my server goes down? at k = 60, the fused list reads:

Table 1: The fused list for the outage question (the module's lab, ranks from 0).

The question as asked reaches neither evidence chunk. The variants reach both, and fusion lifts incidents from 5th place in the union to 3rd. At three chunks sent, recall is 1.00 against 0.00 for the question as asked.

reboot still comes first. Four of the five variants match it, on "server" or on "does" (the chunk says "does not change"), and the fourth variant drifted to How do I restart a server that has stopped responding? We call this agreement on a distractor: RRF counts votes, and votes for the wrong chunk count too. Cut at two chunks and incidents is gone again.

Key distinction: fusion only reorders what the variants retrieved. It cannot add a chunk no variant found, and it cannot tell a relevant agreement from an irrelevant one. A reranker (T6.3) can.

A small k makes rank position dominate: at k = 1, first place is worth 1 and fourth place 0.25. A large k flattens the curve until only the number of votes counts. 03a runs k at 1, 5, 60 and 1000 and notes that the order changes only when rank and agreement disagree.

In the refund question they do. Two variants rank refund-window 1st and 2nd (ranks 1 and 0); three variants rank retention low (ranks 3, 3 and 2). By hand:

Table 2: Rank against agreement, computed by hand.

At k = 60 the three weak votes for retention beat the two strong votes for the evidence chunk. At k = 1 the strong votes win. Mean recall over the three golden questions, from the lab:

Table 3: Mean recall by merge method and cut (five variants per question).

Every merge beats the question as asked. At three chunks every RRF setting beats the union. At two chunks the conventional k = 60 loses to both the union and k = 1, because the refund question drops from 1.00 to 0.50. The value 60 comes from the original RRF paper (Cormack, Clarke and Büttcher, 2009), tuned on other collections; tune k and the cut together on the golden set.

• Merge documents, not ids. RAG-Fusion inherits lesson 2 of T6.2's id-only merge bug if it reuses that merge: key by chunk text, return documents, and assert the context holds passage text.
• Junk variants vote. In a union a junk line only adds noise. In RAG-Fusion each junk query produces a ranked list that votes, so a header such as "Here are five queries:" can lift whichever chunk it matches. Parse with a schema.
• Cost. One LLM call for the five variants, five retrievals run in parallel, and a fusion step that is a dictionary and a sort. The answer call makes it 2 LLM calls and 5 retrievals.

1. Five variants, fused, cut at 3 for questions the router sends here: mean recall 1.00 on the golden set.
2. Tune k with the cut. If the product sends only 2 chunks, k = 1 scores 0.83 and k = 60 scores 0.67 on this set.
3. Watch the distractor. Log each fused list with its votes; reboot winning the outage question shows the variants drifting.
4. Add a reranker (T6.3) over the fused pool when distractors keep winning.

Lesson 4 of T6.2 changes the question in a different direction: instead of rewording it, it asks a broader one.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• RAG-Fusion retrieves for five phrasings and fuses the rankings with RRF, which rewards chunks many variants find; on the Larkspur outage question it lifts incidents from 5th to 3rd.
• Agreement on a distractor still counts (reboot wins with four votes), and fusion only reorders what the variants retrieved.
• The constant k matters when rank and agreement disagree. At two chunks, k = 60 scores 0.67 against 0.83 for k = 1; tune k and the cut together.