Short-Term Memory & Conversation State: overview

A Larkspur customer states their plan, then refers to it in a later question. You will preserve the relevant history for both the model and retriever, isolate sessions and compare full replay with trimming and summaries. The tutorials cover message history, history-aware retrieval, checkpoint threads and summary memory.

The running example is the Larkspur Cloud support assistant answering from the same twelve handbook chunks as the T5.1 lab. Customers ask a first question and then follow up: Can I get a refund on my annual plan? then How long until the money is back?, or How do I export my data? then How long do I have? The lab runs six of these three-turn chats.

• The model is stateless. Turn N of a chat that keeps everything sends 2N − 1 messages, so six turns send 36 messages in total, and cost grows with every turn.
• A session_id (LCEL) or thread_id (LangGraph) picks whose history is loaded. The same chain or graph serves every user; the id is the whole isolation story, so it must come from your server, never from the user.
• Bounding a history is a choice between forgetting and paying: a window or a token trim drops early facts for free, and a running summary keeps them in compressed form for one extra LLM call per fold.
• A follow-up such as How long do I have? names no subject, so the retriever searches for the wrong thing. In the lab at k = 1, sending the latest message finds the evidence on 6 of 12 follow-ups; a rewritten standalone question finds it on 10.
• Appending earlier questions to the query recovers turn 2 (5 of 6) but drowns a change of topic on turn 3 (1 of 6 at k = 1). The rewrite costs an LLM call per follow-up and cannot fix a vocabulary mismatch.

One history feeds two consumers. The rewrite step reads it so the retriever searches for the real subject; the answer step reads it so the model can refer back. After the answer, both messages go into the history, keyed by the session, and the next turn starts from there.

Follow-ups and the retriever: open the file in a browser. Start on The latest message at k = 1 and open the Exports chat: turns 2 and 3 both miss. Switch to Earlier questions + latest and watch turn 2 recover while turn 3 stays stuck on the export chunk. Then pick A rewritten question and compare the three rows of the table, and try k = 2 to see appending catch up by spending a second slot.

• Before: T4.2 builds the LCEL chains that RunnableWithMessageHistory wraps, and T4.4 introduces LangGraph checkpointers. The lab reuses the handbook and TF-IDF index from T5.1; the vocabulary misses it shows are the problem T5.4 solves with embeddings, and T6.2 rewrites single questions the way lesson 4 of T10.1 rewrites follow-ups.
• Next: T10.2 keeps memory across threads: facts about a user that outlive one conversation. T11.3 compacts context at agent-harness scale with the summary idea from lesson 3 of T10.1.

• Memory is the list of messages you resend. Decide on purpose which ones go back, because every one costs tokens on every later turn.
• Key every history by a session or thread id that your server controls, and choose a store that survives a restart before real users arrive.
• Rewrite follow-ups into standalone questions before retrieval, and measure hit rate on turns 2 and 3, not only on first questions.
• Bound long threads by hiding or summarising old turns rather than deleting them, and test that an early fact still reaches the model late in the thread.