Reranking with node postprocessors

What you'll learn

• Explain what a node postprocessor may do to the retrieved list, and when it runs.
• Compare LlamaIndex's LLM reranker and cross-encoder reranker by calls, scores and defaults.
• Trace the Larkspur 45-day question through a cross-encoder, and read a negative score correctly.
• Choose the order of postprocessors when a reranker and a window share the list.

Lesson 2 of T6.6 fused two retrievers and, on the 45-day question, cut the answering sentence in three modes out of four. T6.3 handled that with a second stage. In LlamaIndex the second stage has a fixed slot.

A node postprocessor takes the retrieved list of scored nodes and the question, and returns a list. It can reorder the nodes, drop some, replace their scores or rewrite their text. A query engine runs its postprocessors in list order, after the retriever and before the synthesizer; outside an engine, one can be run by hand on a list of nodes, as 03a does for its comparison.

03a uses the slot for two-stage retrieval (T6.3): retrieve wide with 10 results, rerank, keep 5:

Figure 1: Two-stage retrieval in the postprocessor slot.

The two rerankers in 03a fill that box differently:

Table 1: The two rerankers in 03a.

Two failures follow from the table. The LLM reranker can return fewer nodes than asked, with integer scores that tie often: lesson 3 of T6.3's silent drop and score ties. The cross-encoder's defaults name a different model and a cut of 2, so a port that drops the arguments changes both. We call that the default swap. T6.3 compares the two families in depth (LLM against cross-encoder reranking); in LlamaIndex both swap in the same list.

The cross-encoder reads each node's embedding text, metadata included, so a long file path in metadata is part of every pair it scores (lesson 1 of T6.6).

Split the Larkspur handbook into one node per sentence, ask Can I get a refund on an annual plan after 45 days?, retrieve 10 with all-MiniLM-L6-v2 and rerank with 03a's cross-encoder keeping 2:

Table 2: The 45-day question before and after reranking (local run).

The reranker brings the sentence that answers the question into the context. It also scores that sentence below zero. The ms-marco cross-encoder returns a logit, not a probability, so its scores only rank. We call a cut at zero the negative answer: a filter that drops nodes under 0 would have dropped the answer.

Pitfall: the negative answer. Calibrate any score cut-off on the golden set for the exact reranker you run, or cut by count.

03a builds five chunks: three one-liners that repeat the question's words ("The capital of France is great.", "…huge.", "…beautiful.") and two longer passages about visiting Paris. Asked what is the capital of france?, the embedding retriever ranks the one-liners first. Run locally with the same two models as above:

Table 3: Five chunks under two models (local run).

The cross-encoder reorders the one-liners and keeps all three above the Paris passages. None of the one-liners names Paris, so neither model surfaced the passage that answers the question. The notebook runs this comparison with an LLM reranker and ships without output, so this page does not claim what that reranker returns. An LLM that knows Paris is the capital has knowledge the cross-encoder lacks, which is the case for paying for LLM reranking on questions like this.

The cell that sets up the example also carries a port bug. It builds each Document with LangChain's keyword for the text. LlamaIndex accepts the unknown keyword and creates a Document whose text is empty. The comparison function rebuilds the list with LlamaIndex's own keyword, so the comparison still works, but the first list is five empty Documents, and nothing warns. We call this the silent keyword.

Postprocessors see whatever the one before them produced. With lesson 4 of T6.6's window replacement in the same list, the order decides what the reranker judges:

Table 4: Two orders for a reranker and a window.

Decide which text the reranker should judge, and write the list in that order.

1. Retrieve 10 sentences, rerank, keep 3, passing the model and the cut explicitly.
2. Cut by count, not at a score of zero.
3. Window after reranking for the handbook, so the cross-encoder judges sentences and the model reads windows.
4. Assert Documents have text after building them, to catch the silent keyword.

Lesson 4 of T6.6 adds that window.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• Node postprocessors run in list order between the retriever and the synthesizer, and can reorder, drop, rescore or rewrite nodes. Retrieve wide and let a reranker keep the top few.
• The LLM reranker can silently return fewer nodes with tied integer scores; the cross-encoder reranker defaults to a different model and a cut of 2 (the default swap), so always pass both.
• Cross-encoder scores are logits that only rank: the Larkspur answer scores -2.1011 and still ranks 2nd. Calibrate any cut-off on the golden set, or cut by count.