Vector Stores & Index Operations: interview questions

Production questions of the kind asked in applied AI engineering interviews.

1. Larkspur cut the annual refund window from 30 days to 14 and the nightly sync ran. A week later the assistant still tells some customers "30 days". The new chunk is in the index. What happened, and how do you fix it?

What a strong answer covers:

• Checks the store directly: get() or a search filtered on source="refunds.md" shows both refund chunks, so retrieval picks either one
• Finds the cause in the sync: cleanup=None (it never deletes), or random UUIDs per run, so the old chunk has no ID the pipeline can name
• Fixes it with cleanup="incremental" keyed on source, plus one run to clear the backlog, and checks that the store count equals the folder's chunk count
• Adds the golden question with a negative check: the answer must not contain "30 days"

2. Larkspur wants one assistant to answer from each enterprise customer's private runbooks as well as the public handbook. How do you lay out the vector store, and which store do you pick?

What a strong answer covers:

• Isolation per customer: a namespace per tenant in Pinecone (queries target one namespace), or a tenant key filtered on every query in pgvector, with the filter enforced server-side rather than trusted from the caller
• Picks pgvector when Postgres already holds the customer data (one backup, SQL joins, JSONB metadata) and a managed store when write concurrency or scale outgrows one database
• Keeps the public handbook in its own collection or namespace and merges results, so a handbook re-index never touches tenant data
• Notes that moving between stores is a re-index with the same embedding model, so the choice can wait until load says it must

3. The sync job runs every night. What do you monitor so that you hear about a bad run before a customer does?

What a strong answer covers:

• Logs index()'s four counts per run and alerts on outliers: num_added near the full chunk count means every hash changed (a splitter or metadata change), and a large num_deleted under full means the loader returned a partial folder
• Compares three numbers after each run: chunks the loader produced, record-manager keys, vectors in the store
• Guards full cleanup: refuse to run it when the loader returns fewer files than the previous run, because an empty batch deletes everything
• Re-runs the golden set after any run that added or deleted chunks

4. A small customer's queries come back with zero results even though their documents are indexed. Larger customers are fine. What do you suspect?

What a strong answer covers:

• A filter applied after ranking: LangChain's FAISS wrapper ranks fetch_k=20 candidates and then filters, so a tenant whose chunks are not in the global top 20 gets nothing
• The same effect inside pgvector once an HNSW index exists: filtering runs after the index scan, and the pgvector README's example is 4 matching rows out of 40 candidates when the filter matches 10% of rows
• Fixes: pre-filtering in the store, a larger fetch_k or hnsw.ef_search, iterative index scans (pgvector 0.8.0 and later), or one partition or namespace per tenant
• Logs len(results) against k on every query, since this failure produces no error

5. You switch the embedding model from text-embedding-ada-002 to text-embedding-3-small and re-run the Indexing API over the same folder. What happens, and what should you have done?

What a strong answer covers:

• Nothing is re-embedded: the hash covers text and metadata only, so every chunk is skipped and the store keeps the old vectors
• Both models return 1,536 dimensions, so nothing raises an error; queries embedded with the new model are compared with vectors from the old one and the ranking is noise
• Builds a new collection with its own record-manager namespace, back-fills it, runs the golden set, then switches traffic, or re-embeds in place with force_update=True
• Stores the model name with the collection and asserts it at startup, because a dimension check alone does not catch this