Context enrichment windows

What you'll learn

• Explain how an enrichment window keeps small chunks' precision and large chunks' completeness.
• Compute a window's length from the chunk size and overlap.
• Trace a Larkspur rule cut across a chunk boundary, and the window that rejoins it.
• Diagnose four demo shortcuts that break in production.

Lesson 4 of T6.3 shrank chunks to save budget. Sometimes the chunk is already too small: the answer starts in one chunk and ends in the next.

Larkspur's refund rule shows it. T5.3's 300-character split ends chunk 2 of the refund section at "used less than" and starts chunk 3 with the 100 GB condition. A question about refunds on annual plans retrieves chunk 2, the reranker rightly puts it first, and the model reads a rule with its condition missing. We call this the boundary cut: the ranking is right and the unit of context is too small.

Small chunks retrieve precisely and large chunks answer completely (T5.3). An enrichment window keeps both: index small chunks, and when one is retrieved, send it to the model together with the chunks on either side of it.

Notebook 05a splits the text into fixed-size character chunks and stores each chunk's position in its metadata. The retriever returns one chunk. A window function then fetches the chunks one position before and after it, sorts the three by position and joins them, dropping the overlap at each join so repeated text appears once:

Figure 1: One retrieved chunk and its two neighbours, joined into a window.

Neighbouring chunks start chunk_size - chunk_overlap characters apart, so a window of three chunks covers 2 × (chunk_size - chunk_overlap) + chunk_size characters. 05a runs two examples:

Table 1: Window length at two splits.

The more the chunks already overlap, the less a neighbour adds. Either way the growth comes out of the context budget from lesson 4 of T6.3, so a pipeline that widens each hit usually keeps fewer hits.

For Larkspur's refund section at T5.3's 300-character split, the window around chunk 2 holds chunks 1 to 3, so the 100 GB condition at the start of chunk 3 arrives with the rule it limits.

Here is the same question with and without the window:

Table 2: The boundary cut, with and without a window (illustrative answers).

05a's helpers are written for a demo, and four of their shortcuts break in production:

Table 3: Four demo shortcuts and their fixes.

The last row has a second effect. Two hits next to each other produce overlapping windows, and the model reads the shared chunk twice. We call this window overlap. Merge adjacent windows into one passage; lesson 6 of T6.3 solves that problem in general, by choosing contiguous segments directly.

Interview tip: an enrichment window is a fixed-size answer to a variable-size problem. It grows every hit by the same amount, whether the answer needs one neighbour, none or five.

1. Key chunks by document id and index at ingestion, in a key-value store beside the vector index.
2. Window the top 3 hits, one neighbour each side, and keep fewer hits to pay for them.
3. Merge adjacent windows before building the prompt.
4. Add a boundary-cut golden question (the 100 GB condition) so a change of splitter that moves the cut is caught.

Lesson 6 of T6.3 replaces the fixed window with a segment whose length the reranker's scores decide.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• Index small chunks for precise retrieval, then send each hit with its neighbours so a rule cut across a boundary (the boundary cut) arrives whole.
• A three-chunk window covers 2 × (chunk_size - chunk_overlap) + chunk_size characters: 800 at a 400 / 200 split, 710 at 250 / 20. The growth comes out of the context budget.
• Join with the split's own overlap, key chunks by document id and chunk index, and merge windows from adjacent hits.