Why fine-tune

What you'll learn

• Compare prompting, RAG and fine-tuning by what each one changes, what a change costs and how each one fails.
• Choose between them for a Larkspur request with a three-question decision chain.
• Explain why a base model continues text while a chat model with the same architecture answers it.
• Diagnose the format mismatch that appears when a model is prompted without the wrapper it was trained on.

The modules before this one never touched a model's weights. T3.1 shaped behaviour with instructions and examples, and T5.1 put the Larkspur Cloud handbook in an index so each answer could quote it. This module changes the weights themselves, and the first job is knowing when that is the right tool.

The Larkspur support lead brings two complaints to one meeting. The assistant's answers wander before they reach the point, and it cheerfully discusses What do you think of Mars? when it should decline. The same week, the policy team shortens the refund window. A team that reaches for fine-tuning to fix all three ends up with a model that opens crisply, declines Mars, and quotes the old refund window until someone retrains it. We call that baked-in staleness: a fact trained into the weights that the world has since changed.

Fine-tuning updates the model's weights with your own examples. Prompting changes only the input, and RAG changes only what evidence the input contains. Each one closes a different gap:

Table 1: Three ways to change what a model does, and what each one costs.

For Larkspur the split follows the rows. How long does a refund take? has an answer the policy team will change, so it stays in the index. Leading every answer with Yes or No and declining Mars is behaviour, and a fine-tune can carry it on a smaller, cheaper model. Most production systems that fine-tune also retrieve.

The decision runs as a chain of three questions, each "no" moving one step down:

Figure 1: Prompt first, retrieve facts that change, and fine-tune only behaviour you can show in examples.

The last question carries the most weight. If you cannot write a few hundred examples of the behaviour you want, you cannot fine-tune for it, and you cannot evaluate it either. Run the three Larkspur complaints through the chain and they land in different boxes:

Table 2: The Larkspur complaints routed through Figure 1.

Fine-tuning a behaviour starts from what the model already does before any of your examples arrive, and the notebook shows that starting point directly.

A base model is the network straight out of pre-training: it predicts the next token of whatever text it is given. A chat model is the same network after further training on requests paired with good responses, plus preference training. The notebook sends the same prompts to meta-llama/Llama-2-7b-hf, a base model, and meta-llama/Llama-2-7b-chat-hf, its chat version. The outputs it prints are recorded in the notebook's llama.py:

Table 3: Llama 2 base and chat outputs on the same prompts, as recorded in the notebook's llama.py.

The base model does what pre-training taught it: continue the text with something that looks like more of the same, here a list of similar questions. We call this continuation instead of an answer. The chat model treats the same text as a request.

The [INST] row shows a second, quieter failure. The chat model was trained with that wrapper around every request. With it, the answer starts cleanly; without it, the model first completes the sentence ("on command.") and only then answers. We call this format mismatch: the model sees a prompt shaped differently from its training text and spends its first tokens repairing the shape. A Larkspur model fine-tuned on one template will show the same symptom when the serving code sends another, which lesson 3 of T14.1 takes up.

The architecture did not change between the two columns. Everything that separates them came from training data:

Figure 2: One architecture, two sets of weights. The behaviour gap comes from the training data alone.

That is why the rest of this module is about data: what a training example looks like, how it becomes tokens, how many of them the run sees, and how to tell whether the result beat the prompt it replaces.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• Prompting changes the instructions, RAG changes the evidence and fine-tuning changes the weights. Keep facts that change in the index, and fine-tune for behaviour too long or subtle to describe.
• Try a clear prompt with examples first. Fine-tune only when you can write a few hundred examples of the behaviour, since you need them to train and to evaluate.
• A base model continues text and a chat model answers requests, with the same architecture. The difference comes from training data, and a fine-tuned model expects the format it was trained on.