Inputs and outputs: messages, LLM vs chat model

What you'll learn

• Explain the three message roles and why a chat model keeps no state between calls.
• Trace a conversation as the list the app resends on every turn.
• Compare invoke, batch and generate by what they take and return.
• Diagnose an ignored system rule as a completion model flattening the roles.

Lessons 2 and 3 of T4.1 sent one prompt to a chat model and read one AIMessage back. A support conversation has more than one turn.

A Larkspur customer asks How long does a refund take? and gets a good answer. The follow-up is And for annual plans? The assistant replies as if it had never seen the first question. Nothing failed: the app sent only the second question, and the model had nothing else to read.

A message is one turn of a conversation, tagged with the role of whoever said it. A chat model reads a list of them:

Table 1: The three message classes. Tool results add a fourth, ToolMessage, in T7.1.

The dict spelling works too. LangChain converts each dict to the matching class before the call; we ran the notebook's convert_to_messages on a system, a user and an assistant dict and got a SystemMessage, a HumanMessage and an AIMessage, in that order. Both spellings send the same request.

The conversation is the list

A chat model keeps no state between calls. To continue a conversation, the app appends the model's AIMessage and the next question to the list and sends the whole list again:

Figure 1: Two turns. The second call carries the first turn because the app put it there.

We call the opening bug the forgotten turn: the app sent the new question without the old ones. In notebook 04a, "What did we discuss so far?" gets a real answer only because the earlier turns are in the list. Drop them and the model has nothing to summarise.

Resending has a price. Every earlier turn is paid for again as input tokens on every call:

Table 2: How the list grows. The input cost of turn n includes all n minus 1 earlier exchanges.

History therefore needs a bound (T10.1). It is also just data your app holds, which can be trimmed, summarised or filtered before it is sent. In a template, history goes in a MessagesPlaceholder (lesson 5 of T4.1).

One call or several

For several requests at once, notebook 04a uses two calls besides invoke:

Table 3: Three ways to call a model, from notebook 04a.

With .generate(), each inner list is its own conversation with its own system prompt; the notebook translates one sentence into German and Spanish in one call this way. Each conversation is still its own request to the provider. The model reads roles because it was trained on them, which raises the question of what happens with a model that was not.

LangChain has two model interfaces, and every model class implements one of them. We ran the notebook's base-class check on the installed packages:

Table 4: The two interfaces. A completion model (BaseLLM) continues text; a chat model answers roles.

Both accept both input shapes, because LangChain first turns the input into a PromptValue and then asks it for the shape the model needs:

Figure 2: One input, two conversions. The model's interface decides which one runs.

We ran both conversions with langchain-core 1.7.0 on Larkspur inputs:

Table 5: The two conversions, traced.

A string sent to a chat model becomes one HumanMessage, and nothing is lost. A message list sent to a completion model is flattened into plain text lines, which the model continues like any other text. We call this the flattened role: the system instruction becomes a hint instead of a role the model was trained to follow. That is why every chain in this course uses a chat model, and why a completion model appears only in notebook 04b, for contrast.

Pitfall: the flattened role. A system rule that works on a chat model can be ignored by a completion model given the same messages. Check the base class before blaming the prompt.

For Larkspur's follow-up, the fix is in the app:

1. Keep the conversation as a list of messages per customer session.
2. After each call, append the returned AIMessage, then the next HumanMessage.
3. Send the whole list, through a chat model, so the system rules keep their role.
4. Cap the list at a few turns, or summarise older ones, and watch input tokens per turn.

Lesson 5 of T4.1 moves the list into a template, with the history in a placeholder and the passages and question in their own slots.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• A chat model reads a list of system, human and AI messages and keeps no state. The app resends the history on every call: bound it, and trim or summarise it as data before sending.
• LangChain turns any input into a PromptValue, so a string sent to a chat model becomes one HumanMessage and nothing is lost.
• Use chat models for chains. A completion model flattens messages into plain text lines, and a system instruction becomes a hint instead of a role.