Advanced Indexing Strategies: interview questions

Production questions of the kind asked in applied AI engineering interviews.

1. You switched the Larkspur support bot to parent-document retrieval with no parent splitter. Hit rate went up, but cost per answer tripled and outage answers now quote refund rules. What happened, and what do you change?

What a strong answer covers:

• Without a parent_splitter the parent is the whole document, so every question sends the full page (546 tokens in the lab) whatever the match was
• The model now reads three topics for every question and borrows from the wrong one; the hit rate rose because the answer is somewhere in the page, not because retrieval got sharper
• Adds a parent splitter sized to the handbook's sections (or uses section boundaries from T5.3's structure-aware splitting), then re-measures hit rate, faithfulness and tokens per answer together
• Checks that no parent boundary cuts a golden answer, because a parent that is too small fails the same way a child does

2. The handbook team asks whether to index summaries of each section, small child chunks with their parents, or LLM-generated questions per chunk. How do you choose?

What a strong answer covers:

• Children with parents need no LLM at index time and keep every literal detail searchable; start there
• Summaries help when sections are long and questions are about what a section covers, but a summary can drop the "100 GB" condition, and then no question about it matches
• Generated questions suit short, question-shaped traffic such as support tickets; they cost one LLM call per chunk and must be regenerated when the text changes
• Decides on the golden set with the T5.1 baseline row, one change at a time, and keeps the cheapest key that matches the best result

3. You added contextual chunk headers to the handbook index. How do you show they helped, and how would you notice if they started to hurt?

What a strong answer covers:

• Re-runs the same golden questions against the T5.1 baseline and reports hit rate, answer quality and tokens per answer side by side, as the T5.6 lab brief asks
• Adds questions that name the subject only in the heading ("Do outage credits apply automatically?") because headers help exactly those chunks
• Confirms the reranker and the prompt see the same header text that was embedded
• Watches for headers that dominate short chunks: if many children now score alike because they share a title, precision falls, so header length belongs in the measurement

4. After a nightly re-index the bot answers some questions with no context at all, and a week later other questions return two copies of the same parent. The vector store looks healthy. Where do you look?

What a strong answer covers:

• The docstore and the vector index are two stores tied only by ids; children whose parent ids are missing from the docstore come back as nothing, because the retriever drops None values
• An InMemoryStore docstore is lost on restart while a persistent vector store survives; the docstore needs persistence too (notebook 02b moves it to Postgres)
• add_documents(..., ids=None) mints new UUIDs on every call, so re-running ingestion adds a second set of parents and children instead of replacing the first
• Fixes it with stable ids per parent, deletes of a document's children and parent together, and a check after each run that every child's parent id resolves

5. Contextual retrieval needs an LLM call per chunk with the whole document in the prompt. The handbook has 3,000 pages and changes weekly. Is it worth it, and how do you keep the cost down?

What a strong answer covers:

• Anthropic reports failure reductions of 35% for contextual embeddings and 49% with contextual BM25, on its own datasets; it still has to win on the handbook's golden set
• Prompt caching puts the document in cache once per document instead of once per chunk; Anthropic puts the one-time cost at $1.02 per million document tokens under its stated assumptions
• Regenerates context only for documents that changed, keyed by a content hash, so a weekly update does not re-run 3,000 pages
• Compares against the cheap version first: a header with the document and section title costs no LLM call per chunk and may close most of the gap