Why LLMs forget; classic memory classes

What you'll learn

• Explain why one chat model call cannot see another, and what a message history adds.
• Trace the messages a full-history chat sends on each turn, and do the arithmetic on its cost.
• Diagnose a half-saved history and the missing save.
• Map LangChain's classic memory classes to the patterns that replaced them.

T10 starts here. Earlier modules built chains and agents that answered one question at a time. From this lesson on, the Larkspur Cloud support assistant has to hold a conversation.

A customer writes I'm on the annual Team plan and gets a friendly reply. Their next message is Which plan am I on?, and the assistant has no idea. Nothing crashed and the model works. The second call carried only the second question.

A chat model call is a function of the messages sent in that call and nothing else. The provider keeps nothing between calls, so a stateless chain of prompt, model and parser treats every message as the first one. The notebook shows the same thing with a name: told My name is Sourav in one call, the chain cannot answer What's my name? in the next. Anything that looks like memory in a chatbot is the application resending earlier messages.

A message history is the ordered list of typed messages that make up the conversation so far, and the application sends it with every call. The model then reads the earlier turns as part of the current input, exactly as it reads the new question.

Three message types share the list:

Table 1: The three message types. Because the prompt template adds the system message each time, the stored history holds only the human and AI turns.

Each turn appends the user's message, sends the whole list and appends the reply. Figure 1 shows three turns:

Figure 1: Full-history replay. Every call carries every earlier message plus the new one.

The notebook's hand-rolled version does four things per turn: append the human message, call the model with the whole list, append the reply, return it. LangChain gives the same list a home in InMemoryChatMessageHistory, with operations to add a user message, add an AI message, read the messages and clear them. File, SQLite and Redis backends implement that same interface, and lesson 2 of T10.1 relies on it.

What full replay costs

A history that keeps everything resends everything. Turn N carries the N questions so far and the N − 1 replies before it, which is 2N − 1 messages:

Table 2: Messages sent by a full-history chat over six turns, not counting the system message. The running total is N squared.

Six turns send 36 messages where six independent calls would send 6. Every earlier message is read again on every later turn, and token cost and latency follow that curve. A long enough chat stops fitting in the context window at all. We will call this replay cost; lessons 3 and 6 of T10.1 bound it.

Planted failures: what the second call receives

Having a history is half the job. Which messages get into it decides what the model can use. Table 3 replays the Larkspur exchange with the history kept in three ways:

Table 3: Three histories for the same two turns. Only the first gives the model a faithful transcript.

The third row is a half-saved history. It looks healthy in a demo, because the customer's facts survive. It fails later, when the customer asks What did you just tell me about refunds? and the reply that answered it was never stored. The older memory classes had a worse version of this bug, as the next section shows.

Before RunnableWithMessageHistory and LangGraph checkpointers, LangChain packaged memory as classes that a chain loaded before each call and saved after it. An LCEL chain has no memory slot. The optional notebook drives them through a loop of three steps:

Figure 2: The load, invoke and save loop. The chain never writes to memory; the caller's save step does.

The save is the step people forget. Skip it and the chain still runs without an error, but the next load returns nothing and every turn behaves like the first. This is the missing save. In the healthy case, the notebook asks What are the first four colors of a rainbow? and then And the other 3?. The second question only makes sense because the first exchange was loaded, and after two turns memory holds 4 messages, both pairs written by the explicit save.

Each class decides which messages the prompt receives. Table 4 lines them up with the patterns that replaced them:

Table 4: The five classic classes. Each one is a policy for choosing which earlier messages to resend.

Three details from the notebook are worth carrying forward:

• The window hides old turns without deleting them. The window class still stores every message underneath and returns only the last k exchanges. With k = 1, a favourite colour given in the first exchange is invisible by the third question, although it is still stored.
• Choosing k is a guess about the conversation. The notebook suggests 1 to 2 exchanges for quick questions, 3 to 5 for support chats and 6 to 10 for long technical threads. Whatever the value, early facts leave without warning.
• The format is separate from the policy. A memory can hand the prompt one "Human: … AI: …" string for completion-style prompts or a list of message objects for chat prompts. The choice changes formatting and leaves what is remembered alone.

These classes now live in langchain_classic. You will meet them in older code; new code uses the patterns in lessons 2 to 6 of T10.1.

Following the Larkspur customer through a full-history chat:

1. Turn 1. The app appends I'm on the annual Team plan, sends one message and appends the reply. The history holds 2 messages.
2. Turn 2. The app appends Which plan am I on? and sends 3 messages. The model reads the plan from Human 1 and answers.
3. Turn 6. The same chat now sends 11 messages per call and has sent 36 in total. Each message the customer adds will be paid for again on every later turn.

What changes next: one list works for one customer. Lesson 2 of T10.1 keys a history to each conversation, and the same chain then serves every Larkspur customer without mixing their turns.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• A chat model call sees only the messages sent in that call. Keep an ordered history of human and AI turns, send the whole list each time, and let the prompt add the system message.
• A history that keeps everything resends everything: turn N sends 2N − 1 messages, and six turns send 36 against 6 for independent calls. That replay cost is what later lessons bound.
• The classic memory classes load before each call and save after it. A missing save leaves every turn with an empty history and raises no error; expect these classes in older code only.