RAPTOR: clustering and recursive summarisation

What you'll learn

• Explain why a passage index struggles with a question about a theme that spans several sections.
• Trace how RAPTOR builds summary levels by clustering, summarising and repeating.
• Compare collapsed-tree retrieval with top-down tree traversal under the same context budget.
• Diagnose a summary that reaches the right sources and still drops the condition a customer asked about.

T5.1 built the flat baseline: split the handbook into passages, score each passage against the question, keep the top few. That works when one passage holds the answer. This module asks what to do when no single passage does.

An operations lead at Larkspur Cloud asks How does service continuity work? The handbook never answers that in one place. Availability guarantees and outage credits sit in the SLA section, status updates in the outages section, and data protection in export and retention. A flat index scores each of those passages against the word "continuity", and most of them never use it.

The failure is a scoring problem. A theme is spread across passages, and no passage is about the theme as a whole. RAPTOR (Recursive Abstractive Processing for Tree-Organised Retrieval) answers it by adding new text to the index: summaries that are about the theme. The rest of this page covers how those summaries are built, how retrieval searches them, and how they mislead.

An abstraction level is one layer of text in the index, from the original passages at level 0 up through summaries of summaries. Each level answers a different kind of question. A passage keeps the exact wording; a summary connects claims across passages; a summary of summaries names a broad theme.

The same handbook at three levels looks like this:

Table 1: One handbook at three abstraction levels.

RAPTOR builds the levels bottom up. It embeds the current texts, groups them into clusters by meaning, writes one summary per cluster, then treats those summaries as the texts for the next round:

Figure 1: The RAPTOR build loop. Each round clusters the current level and summarises every cluster into the next one.

The notebook builds this loop over a 33-page climate report, one page per level-0 text. It clusters with a Gaussian mixture model, asks a chat model to summarise each cluster, and stops after at most three rounds. The number of clusters per round is half the number of texts, capped at 10. That rule sets an upper bound on the tree's shape:

Table 2: The tree's shape from the notebook's cluster rule. Actual counts can be lower when a mixture component receives no text.

The notebook assigns each text to exactly one cluster. That is a teaching simplification: the RAPTOR paper (Sarthi et al., 2024) uses soft clustering, where one passage can feed several summaries, and it describes both search strategies covered in the next section.

The lab's fixture is the smallest tree that shows the idea. It has four authored passages, two level-1 summaries and one level-2 summary, with each summary listing the source passages it descends from:

Figure 2: The lab's fixture tree. The summaries are written by hand and labelled representative; no clustering runs in the browser.

Each node needs more than its text. Store its level, the identifiers and versions of the passages it summarises, and its children. With those fields a retrieved summary can lead back to its evidence, and a changed passage can mark every ancestor for rebuilding.

A summary can be accurate overall and still drop the one exception a customer asks about. Call that lossy compression: the summary's sources cover the policy, but its text no longer carries the condition. For a policy answer, follow the summary back to its passages before quoting a condition or a number.

The notebook's metadata does not support that walk yet. Its level-0 records hold level, origin and parent_id but no id, and each summary builds its child_ids by reading that missing field. Every first-level summary therefore points at children with no identifier. Treat the notebook's tree as a sketch of the structure, not as verified provenance.

With levels in place, the next question is how retrieval chooses among them.

Collapsed-tree retrieval puts every node, passages and summaries alike, into one pool, scores them all against the question and keeps the best within a budget. The mix of levels in the context then follows the question: a theme question pulls summaries, a procedure question pulls passages.

Tree traversal works the other way. It picks the best nodes at the top level, then searches only their children, and so on down. One early wrong branch excludes every leaf beneath the other branches:

Figure 3: Two ways to search a tree. Collapsed search ranks every level at once; traversal commits to parents before it sees their children.

The notebook stores every level in one FAISS index, which is what makes collapsed search possible. Its query helper then walks the levels from the top down, passing a level filter that FAISS does apply, and returns up to four nodes per level. After each level it appends the text AND id:(…) with the chosen children to the query string. That text changes the query that gets embedded; it does not filter by identifier or join parents to children. Check the identifiers of the nodes you get back before calling this a constrained traversal. A compression step then rewrites each retrieved node to the parts a model judges relevant, one more transformation whose output needs a source check.

The notebook also defines a proper traversal function that filters children by parent identifier. It is never called, and it would fail as written: it calls a FAISS method that the installed library does not provide.

The interactive lab isolates the one change that matters here: adding summary nodes to the pool. It scores the fixture of Figure 2 with real TF-IDF cosine similarity, one vocabulary fitted over all seven nodes. Leaf-only mode keeps level 0; collapsed mode keeps everything.

At a budget of two nodes, the lab computes these results for its first two questions:

Table 3: Leaf-only against collapsed retrieval at two nodes. Scores checked against scikit-learn's TfidfVectorizer.

For continuity, the only passage that matches is availability, at a score of 0.168. With summaries in the pool, the handbook summary (0.448) and the continuity summary (0.378) take both places and reach all four source identifiers. For export steps, the export passage scores 0.387 and no summary makes the top two, leaving the context unchanged.

Reaching four source identifiers describes ancestry. It does not mean the context holds the source passages, and it says nothing about the answer a model would write. The lab's third question shows the gap:

Table 4: Planted failure in the lab. The summary that enters has no "30 days" in its text.

The retention passage still ranks first, and the answer survives here. The pattern is what to watch: a summary that matches the question's vocabulary takes a slot a passage held. Call it summary displacement. At a tighter budget, or with a vaguer passage, the summary would be the only evidence, and lossy compression would decide the answer.

The notebook build has its own planted failures, one per guard it is missing:

Table 5: Planted failures in the notebook's RAPTOR build.

These are fixes to make before trusting the tree, and none of them changes the idea. Collapsed search over a tree with real provenance is the design worth copying.

A Larkspur support team adding RAPTOR to the handbook would work through it in this order:

1. Keep the passage index as level 0, with stable identifiers and versions.
2. Build summaries per cluster and store each one's level, sources, versions and children.
3. Search all levels together at a fixed context budget, and log which level each hit came from.
4. When a summary ranks high, add its source passages to the context before the model quotes a condition.
5. When a passage changes, rebuild every summary above it.

To test it, freeze the corpus and a question set, then compare flat retrieval, the summary tree and graph retrieval at the same final budget. Count indexing calls apart from retrieval and generation calls. Include precise facts, cross-section comparisons, changed policies and questions the handbook cannot answer.

Lesson 2 of T13.1 replaces the summary tree with a different derived representation: a memory of the corpus that writes search clues.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• RAPTOR clusters passages, summarises each cluster and repeats, so one index holds both exact wording and broad themes. Store each node's level, sources, versions and children with its text.
• Collapsed-tree retrieval ranks all levels in one pool and lets the mix follow the question; top-down traversal can lose a relevant leaf to an early wrong branch.
• A summary can reach the right sources and still drop the condition asked about. Follow it back to its passages before quoting a condition or number, and watch for summaries displacing passages at a tight budget.