LCEL, Runnables & Chains: interview questions

Production questions of the kind asked in applied AI engineering interviews.

1. Larkspur's nightly job runs triage_chain.batch() over 2,000 support tickets. Last night it died at ticket 1,412 with a rate-limit error and wrote nothing. What do you change?

What a strong answer covers:

• Caps parallel calls with config={"max_concurrency": n} sized to the provider's rate limit, since the default is Python's thread-pool size
• Wraps the model step in .with_retry() so a transient 429 retries that call instead of failing the batch
• Passes return_exceptions=True so one bad ticket returns an exception in its slot and the other 1,999 results survive
• Writes results in pages (or checkpoints them) and adds .with_fallbacks([...]) to a second model if outages are common

2. Product wants a weekly digest of every handbook change. Some weeks one paragraph changes; in release weeks, 40 pages do. Which summarisation strategy do you build?

What a strong answer covers:

• A RunnableBranch on input length: stuff below a token budget (one call, nothing summarised twice), map-reduce above it
• Map-reduce for the long case: the map step parallelises, so latency is about one map round plus the reduce, and every chunk is treated the same
• Refine only if later changes should override earlier ones, accepting one sequential call per chunk and order sensitivity
• A guard for the reduce step itself: if the joined summaries exceed the budget, reduce in groups and reduce again

3. The support chain routes tickets with a classifier and a RunnableBranch. How do you find out, in production, which branch each ticket took and which branch is slow?

What a strong answer covers:

• Traces every run (LangSmith or any callback handler); a pipe records each step as a child run with its own latency
• Names the steps with .with_config(run_name=...) and tags runs with metadata such as ticket id and plan, so traces can be filtered
• Logs the classifier's raw output next to the chosen branch, since a wrong route usually starts as an unexpected label
• Dashboards the share of tickets per branch and the default-branch rate, which rises when the classifier drifts

4. The ticket classifier returns "Refund." for most refund tickets, but for one ticket it returns "Refund or outage". The branch conditions are "refund" in label and "outage" in label. What happens, and how do you make routing safe?

What a strong answer covers:

• RunnableBranch tests conditions in order and runs the first true one, so the ticket silently goes to the refund chain
• Substring checks on free text are fragile: punctuation, case and multi-label answers all route somewhere
• Constrains the classifier with structured output (an enum of labels) and validates it before the branch
• Keeps a default branch that escalates to a human, and counts how often it fires

5. You inherit a Larkspur service built on RetrievalQA, LLMChain and hub.pull("rlm/rag-prompt") on LangChain 0.x. How do you migrate it to 1.x without breaking the API that callers use?

What a strong answer covers:

• Repoints imports to langchain_classic first so the service runs on 1.x, then rewrites chain by chain behind the same interface
• Replaces RetrievalQA with {"context": retriever | format_docs, "question": RunnablePassthrough()} | prompt | llm | StrOutputParser(), and maps the output shape: the legacy chain returns a dict with query and result, the pipe returns a string
• Inlines the prompt, because hub.pull is deprecated and raises for public prompts on the repo's pinned langsmith
• Replaces every chain(...) and .run() call with .invoke(), and compares old and new answers on a fixed question set before switching