Query Transformation & Routing: interview questions

Production questions of the kind asked in applied AI engineering interviews.

1. You ship RAG-Fusion for Larkspur's support assistant. A customer asks "What happens if my server goes down?" and gets reboot instructions instead of the SLA credit policy. Recall on the golden set went up. What happened?

What a strong answer covers:

• Prints the fused list with each chunk's votes and ranks: the reboot chunk can win because four of five variants matched "server" or "does", which is agreement on a distractor
• Checks the variants themselves; a variant that drifts ("How do I restart a server…") adds a vote for the wrong chunk
• Looks at the cut: at three chunks both evidence chunks are in, at two only sla is, so the number of chunks sent matters as much as the fusion
• Fixes with a reranker over the fused pool (T6.3), a lower k or fewer drifting variants, and adds the question to the golden set at the production cut

2. Your latency budget is 1.5 s and a plain retrieval takes 300 ms. Product wants multi-query, HyDE and decomposition "on every question". What do you propose?

What a strong answer covers:

• Counts the calls: multi-query adds one LLM call and N retrievals, HyDE one LLM call, decomposition five LLM calls and three retrievals where plain RAG has one and one
• Routes first: send compound questions to decomposition, short or vague ones to variants or HyDE, and direct lookups straight to retrieval
• Runs the per-variant retrievals and sub-questions in parallel (retriever.map(), .batch()), and uses a small model for rewriting
• Measures each technique per question type against the T5.1 baseline and keeps only the ones that win where they are applied

3. How do you show that RAG-Fusion helped, rather than only cost more?

What a strong answer covers:

• Compares against the plain pipeline on the same frozen golden set, at the cut the product really uses
• Reports recall at that cut, answer quality, latency and LLM cost per question, and breaks results down by question type
• Watches regressions on direct lookups: in the lab, RRF at k = 60 and two chunks drops the refund question from 1.00 (union) to 0.50
• Logs the variants and fused lists in production so a bad variant or a distractor that keeps winning shows up in traces

4. Larkspur routes questions semantically to billing, data, reliability and servers. "Do I get a refund if an outage lasts a day?" goes to servers. How do you make the router safer?

What a strong answer covers:

• Reads all route scores and the margin, not only the winner; a question that straddles billing and reliability will have close scores
• Adds a minimum top score and a minimum margin, tuned on logged traffic, and sends anything below either to an LLM classifier or a default prompt
• Gives the classifier an explicit "other" destination, because a Literal guarantees a valid route, not a sensible one
• Adds exemplars for the cross-topic phrasings that fail, and keeps a regression set of query to expected route

5. A self-querying retriever turns "annual-plan refund rules updated after 2025" into a metadata filter. Some days it returns nothing, and the logs show no error. What do you check?

What a strong answer covers:

• Logs the structured query each call produced, because the query constructor is an LLM and its output varies between runs
• Looks for a filter value of the wrong type ("2025" as a string), which a store such as Chroma rejects for $gt, and coerces numeric values before they reach the store
• Checks the metadata itself: an attribute that is not declared cannot be filtered, and "annual" against "Annual" breaks equality
• Treats an empty result after a filter as a signal to retry without the filter or to ask the user, not as "nothing matched"