Fine-Tuning & Model Adaptation: overview

Fine-tuning changes a model’s weights with your own examples, so the change travels with the model instead of sitting in every prompt. It is good at behaviour (a format, a tone, a refusal, one narrow task) and poor at facts that change. The T14.1 notebooks build on four ideas: how instruction data turns a text predictor into an assistant, how that data becomes token tensors, what the Hugging Face Trainer does with them, and how to tell whether the result is better.

The notebooks fine-tune EleutherAI/pythia-70m on lamini_docs, 1,400 questions and answers about one company’s product. These pages run the same steps on the Larkspur Cloud support assistant from T5.1 to T5.3: the support team wants answers that lead with Yes or No, state the policy in a sentence, and decline questions that are not about Larkspur. The examples reuse T5.1’s golden questions (How long does a refund take?, Can I export my data after I cancel my plan?, What happens if my server goes down?) and add the notebooks’ off-topic probe, What do you think of Mars?

• Fine-tune for behaviour, retrieve for facts. The notebooks’ clearest result is a format change: the base Llama 2 continues “Tell me how to train my dog to sit” with more questions, and the chat model answers it. A fact that changes next month belongs in the T5.1 index.
• The training text is all the model learns from. The lamini/lamini_docs copy that lesson 5 of T14.1 trains on joins each question to its answer with no template and no end token, in all 1,260 training rows. A model trained that way never learns where an answer stops.
• Truncation and padding decide what survives. In the lab, max_length = 48 cuts 11 answer tokens from the refund example with right truncation, or the header and 7 question tokens with left truncation. Padding the batch to its longest row adds 18 pad tokens, 13% of the batch.
• Count examples, not steps. The lesson 5 of T14.1 run is max_steps = 3 with a batch of 1 and 4 accumulation steps: 12 of 1,260 training examples, under 1% of one epoch, and the first update runs at learning rate 0.
• Exact match fails on free text. The notebooks’ evaluation set has 0 exact matches in 139 answers, and 127 predicted answers repeat a sentence. Measure the behaviour you trained for, on held-out questions, against the prompted or RAG baseline.

When evaluation shows a problem, the fix is usually in the examples or in how they were prepared (a missing end token, a truncated answer, a template the model never sees at inference), and the loop goes back to the data before it goes back to the training settings.

Truncation and padding: open the file in a browser. Start at max_length = 48 with right truncation and watch the refund answer lose its last 11 tokens; switch to left truncation and the ### Question: header goes instead. Then set the end token to Appended, slide to 64 so every row ends with it, and pick To max_length to see padding rise to 27% of the batch.

• Before: the whole course up to T13.3. T2.3 loads Hugging Face models and tokenizers, which every T14.1 notebook uses. T3.1 gets as far as prompting can go, which is the first thing to try before fine-tuning. T5.1 puts facts in an index instead of the weights, and its baseline is what the T14.1 lab compares a fine-tuned model against.
• Next: T15.1 chooses the capstone architecture, and its design document takes the RAG or fine-tune row from the T14.1 lab verdict.

• Try prompting first, put changing facts in retrieval, and fine-tune for behaviour you can describe and measure.
• Before training, decode a few prepared rows: check the template, the end token, what truncation removed and what padding added.
• Report training runs in examples and epochs, and read the learning-rate schedule for the steps you actually run.
• Evaluate the fine-tuned model on held-out questions, on the behaviour you trained for, against the baseline it is meant to replace.