Commercial LLMs natively and with LangChain

What you'll learn

• Explain what a LangChain chat model adds over a provider's own SDK.
• Read the three parts of an AIMessage: content, token usage and provider metadata.
• Diagnose a reply that stopped at the token limit from its metadata.
• Decide when a provider swap is ready to ship.

Lesson 1 of T4.1 placed the chat model in a partner package, the one part of a chain that lives outside langchain-core. This lesson looks at what that class gives you.

Larkspur's support team wants to compare OpenAI and Gemini on the refund question. The optional notebook calls both through their own SDKs, and the two calls share almost nothing:

Table 1: The same request in the two native SDKs, from the optional notebook.

Every line that touches the response is provider-specific. Supporting a third provider means a third set of those lines, and every place that logs tokens or checks why a reply stopped has to know which provider answered. A common interface fixes that.

A chat model is any class that implements LangChain's BaseChatModel interface: it takes messages and returns an AIMessage, whichever provider answers. The provider SDK still makes the call underneath; the class translates the request on the way out and the response on the way back:

Figure 1: A chat model wraps the provider's call. The chain sees one response type.

Every chat model has the same three calls:

Table 2: The three calls every chat model offers.

What comes back

The AIMessage carries more than the text. Notebook 02a reads three attributes:

Table 3: The three parts of an AIMessage that lesson 2 of T4.1 reads.

The last row matters more than it looks. On OpenAI, a finish_reason of length means the reply hit the token limit and stopped mid-sentence, or mid-JSON. We call this the truncated reply: the text reads like an answer and ends early. Lesson 6 of T4.1 shows what each output parser makes of one.

The notebook also takes the chain apart. prompt | model builds a RunnableSequence, and so do prompt.pipe(model) and RunnableSequence(prompt, model); we ran all three with a stub model and each gave a sequence of 2 steps. Calling the template on its own returns a ChatPromptValue holding one HumanMessage ("Explain AI in 2 bullet points"), which is exactly what the model then receives. Invoking each step by hand is how you debug a prompt.

Changing provider

Changing provider changes one class. In the notebooks the same prompt | model chain runs on all of these:

Table 4: One chain, five model classes. The last three are lesson 3 of T4.1's open-source backends.

Lesson 3 of T4.1 runs those three. ChatHuggingFace is a wrapper: the two Hugging Face classes underneath are completion-style LLMs, and the wrapper gives them the chat interface (lesson 4 of T4.1).

init_chat_model from langchain moves the choice out of the import line and into a string, which is how notebook 01b loads its model. The string names the provider and the model together:

Table 5: Two init_chat_model strings from notebook 01b. Swapping provider is an edit to the string.

Planted failure: the silent provider swap

The swap keeps the code running. It does not keep the replies the same: another model formats, refuses and rambles differently. We call this the silent provider swap: every call succeeds, and the answers drift. For Larkspur, the refund question might come back with the timing right and the citation format wrong, which no exception reports. A provider change is a release like any other, checked against the golden questions from T5.1 before it ships.

For the Larkspur comparison, the chat model reduces the work to this:

1. Build one chain from the answer template and a chat model named by an init_chat_model string.
2. Run the golden questions through it once per provider string.
3. Compare .content against the expected answers, .usage_metadata for cost, and finish_reason in .response_metadata for truncated replies.
4. Ship the provider that passes, and keep the string in configuration so a rollback is one edit.

Interview tip: "LangChain makes models interchangeable" is half true. The call shape is interchangeable. The behaviour is not, and a strong answer says how you would measure the difference.

Lesson 3 of T4.1 runs the same chain on open-source models, locally and hosted.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• Every LangChain chat model offers invoke, batch and stream and returns an AIMessage, with token counts under the same keys for every provider.
• Read .response_metadata as well as .content. A finish_reason of length means the reply hit the token limit and stopped mid-sentence or mid-JSON.
• Changing provider changes one class or one init_chat_model string, but the replies change too. Treat a provider swap as a release and check it against the golden questions.