Where the splitter cuts decides what the retriever can find.

LangChain’s RecursiveCharacterTextSplitter, ported line for line, runs here on one handbook section. Move chunk size and overlap and check whether the answer to the golden question survives in one piece.

Golden question

How long do I have to get a full refund on an annual plan?

The answer, as the handbook states it

chunk_size, in characters

chunk_overlap, in characters

Overlap is capped at half the chunk size. Separators are ["\n\n", "\n", " ", ""], and length counts characters, the LangChain default.

Chunks
Stored text over source text
Answer in one chunk
Answer’s share of its chunk

Chunk map of the 1,111-character section

chunk rangetext stored twice (overlap)the answer sentence

The chunks

Answer marked; repeated text shaded