Python & Data Toolkit Refresher: interview questions

Production questions of the kind asked in applied AI engineering interviews.

1. Larkspur's nightly evaluation sends 200 golden questions to the model. A teammate rewrote it with asyncio.gather to make it faster, and it takes exactly as long as before. What do you look for?

What a strong answer covers:

• Checks whether the coroutine calls a sync client (llm.invoke, requests.get, time.sleep); a call that never awaits holds the event loop, so the tasks run one after another
• Fixes it with the async method (ainvoke, aiohttp) or by moving the sync call to a thread with asyncio.to_thread
• Confirms with timestamps per call: overlapping start times mean it is concurrent, back-to-back ones mean it is not
• Adds a semaphore sized to the provider's rate limit before the fix sends 200 requests at once

2. You need to embed 50,000 handbook chunks through a hosted embeddings API. Threads, processes or asyncio, and what limits the speed?

What a strong answer covers:

• The work is network waiting, so asyncio (or threads with a sync SDK) fits; processes add cost and solve a CPU problem this job does not have
• Sends several chunks per request when the API accepts a list, which cuts the number of calls before concurrency matters
• Caps in-flight requests with a semaphore at the provider's rate limit, and retries 429 and 5xx responses with backoff
• Measures throughput at a few limits and stops raising it when wall time stops falling, as it does at 7 in the lab

3. Retrieval scores computed in a notebook with NumPy disagree with the cosine scores the vector store returns for the same Larkspur chunks. How do you find out which is wrong?

What a strong answer covers:

• Asserts that every row norm is 1.0 after normalising (np.allclose(np.linalg.norm(E_n, axis=1), 1))
• Checks the shape of the norms: without keepdims=True a square matrix divides columns and gives wrong scores with no error
• Compares a few rows against sklearn.metrics.pairwise.cosine_similarity and checks whether the store reports distance or similarity
• Checks dtypes and that the same embedding model and dimension built both indexes

4. The support assistant looks up a customer's support response time with SELECT support_hours FROM plans WHERE name = '{plan}', where plan comes from the account record. What is wrong, and what else would you change?

What a strong answer covers:

• String-built SQL is injectable: starter' OR '1'='1 returns every row; pass the value as a parameter (? in sqlite3, %s in psycopg2)
• Opens the connection read-only or with a user that can only SELECT from the tables the assistant needs
• Reads credentials from the environment, never from a notebook cell
• Handles the no-row case (fetchone() returns None) with a fallback answer instead of an exception

5. The weekly report shows mean recall@2 rising from 0.50 to 0.75 after a retriever change. Before celebrating, what do you check in the results DataFrame?

What a strong answer covers:

• Compares count() with size() per group: mean() skips NaN, so a question that errored drops out and lifts the average
• Checks that both runs used the same golden questions (a merge on question id, not on row order)
• Looks at per-question rows with pivot_table, since one question type can regress while the mean improves
• Records failures as an explicit status column so the next report counts them