Architecture Synthesis: Choosing a System Design: interview questions

Production questions of the kind asked in applied AI engineering interviews.

1. Larkspur's support assistant moved from plain RAG to Self-RAG. Latency is several times higher, and “What happens if my server goes down?” still gets a wrong answer. What happened, and what do you change?

What a strong answer covers:

• Prints the retrieved chunks per pass: TF-IDF returns reboot, sizing and export every time, so the checks see the gap and loop over the same chunks (14 calls in the T9.2 lab at k = 3)
• Names the failing stage as retrieval: the handbook says "outage" and "availability", the question says "goes down"
• Fixes retrieval first (dense ranks both evidence chunks first and second in the T6.1 lab, at one call) and keeps a check only where a wrong answer is costly
• Caps every loop in code and logs model calls per question, so a later regression shows up in the logs before it reaches the bill

2. The product team wants billing, operations and data specialists. When is a router enough, when do you need a supervisor, and when a swarm?

What a strong answer covers:

• Measures how many requests span two domains; a router (2 calls) sends My server went down last night. Do I get service credits? to operations only, and billing never sees the credits question
• A supervisor answers it in 5 calls, 3 of them routing, and gives one place to log, test and cap every decision
• A swarm answers it in 3 and each follow-up costs 1 call against 3 for the supervisor, but routing quality now lives in every agent's prompt
• Keeps plain RAG over all three knowledge bases as the baseline: the export question needs one call whatever the data is split into

3. Leadership asks whether GraphRAG is worth replacing the flat index. How do you measure it?

What a strong answer covers:

• Runs both on the same golden set and budget, with questions of each kind: a specific fact, a theme across the handbook, a relationship such as EU availability
• Reports per stage and per question: context recall shows what reached the model, answer correctness what came out, and a mean can hide one question type that got worse
• Counts index-time work separately from query time: summary and extraction calls, re-indexing after a handbook edit, storage
• Treats lab fixture counts (T13.1's 1 against 4 source passages, T13.3's 2 against 4 relations) as mechanisms to test; neither lab measured answer quality

4. A customer asks “Does Larkspur offer GPU servers?” The handbook does not say. How do you make sure the assistant refuses instead of guessing?

What a strong answer covers:

• Plain RAG still sends the top k chunks, all unrelated, so only the prompt and a permission to say "I don't know" stand between them and a guess
• A relevance grader with a bounded rewrite loop ends in a no-results reply, at 9 calls with two rewrites allowed (T9.1); a score floor is cheaper if the golden set supports one
• CRAG's web fallback cannot see a private handbook, and the T9.2 Self-RAG graph generates with no context and skips the groundedness check on that path
• Adds unanswerable questions to the golden set and tracks the refusal rate, so a prompt change that starts guessing fails a test

5. A teammate proposes replacing the dense retriever with ColBERT. What do you check before agreeing?

What a strong answer covers:

• Storage and indexing: one vector per token instead of one per chunk, up to 23,040 numbers per 180-token passage before compression against 1,536
• The questions that fail today and why: exact codes such as E-4021, where dense ranked the right chunk 5th in T6.1, may be cheaper to fix with BM25 in a hybrid
• A cross-encoder reranker (T6.3) reads question and chunk together at query time; ColBERT precomputes document vectors, so compare both on recall and latency, not one in isolation
• Model calls do not change, so the decision rests on retrieval recall, latency and storage measured on the golden set