LangChain Core: Models, Prompts & Parsers: interview questions

Production questions of the kind asked in applied AI engineering interviews.

1. Larkspur's refund answers start failing: about one request in fifty raises OutputParserException after a model upgrade. The previous model never did. Where do you look first?

What a strong answer covers:

• Pulls the raw AIMessage.content for failed requests before changing any code; the exception's llm_output holds what the parser saw
• Sorts the failures by shape: prose before the JSON, a truncated reply (finish_reason of length in response_metadata), a missing field, a wrong type
• Fixes each shape at its cause: a token limit for truncation, the schema description for a wrong type, with_structured_output where the provider supports it
• Adds the failing replies to a parser test set, so the next upgrade is checked against them before it ships

2. The answer must cite the passages it used. A teammate proposes JsonOutputParser because it "never breaks". Do you agree?

What a strong answer covers:

• JsonOutputParser rarely raises because it repairs text: it strips fences, drops trailing prose and closes truncated JSON, so a cut-off reply arrives as a dict with a half-written citation
• An exception is the cheaper failure: it can be retried or logged, while a wrong dict reaches the customer
• Puts the rules in a schema: citations as a list of the retrieved passage ids (a Literal or a post-parse check), answer as a required string
• Keeps a fallback for the raise, as the lesson 7 of T4.1 bot does, and counts how often it fires

3. How do you monitor the structured-answer step in production?

What a strong answer covers:

• Rate of OutputParserException per model and per prompt version, with the raw reply logged for each one
• Rate of answers whose citations are not among the retrieved passage ids, checked in code after parsing
• Token counts from usage_metadata, split into input and output, because format instructions and history grow the input side
• Share of replies with finish_reason of length, which predicts truncated JSON before the parser sees it

4. A customer asks "How long does a refund take?" and then "And for annual plans?". The second answer ignores the first turn. What happened, and how do you fix it without the prompt growing forever?

What a strong answer covers:

• Chat models keep no state; the second call saw only the second question unless the app resent the first turn and the reply
• Puts history in a MessagesPlaceholder and appends the AIMessage after each turn
• Bounds the history: keep the last few turns, or summarise older ones (T10.1), and measures the input-token cost per turn
• Rewrites the follow-up into a standalone question before retrieval, because the retriever needs "annual plans" and "refund" in one query (T10.1)

5. You move the refund assistant from gpt-4o-mini to a Groq-hosted open model to cut cost. The chain code does not change. What can still break?

What a strong answer covers:

• The call shape survives a swap (init_chat_model("groq:…") or ChatGroq), the replies do not: format compliance, refusal style and verbosity can all change
• with_structured_output depends on the provider's structured-output or tool-calling support; check it before relying on it, or fall back to a parser
• Re-runs the golden questions from T5.1 and the parser test set, and compares usable-answer rate, citation validity, tokens and latency
• Keeps the provider in configuration (a "provider:model" string) so a rollback is a config change