Fine-Tuning & Model Adaptation: interview questions

Production questions of the kind asked in applied AI engineering interviews.

1. Larkspur fine-tuned a small model on 2,000 support transcripts. In testing it answers the refund question correctly, then keeps going: it repeats the policy sentence until the response hits its token limit. What do you check?

What a strong answer covers:

• Decodes a few training rows and checks that each ends with the tokenizer’s end-of-text token; the Pythia tokenizer does not append one, so a model trained on raw question-plus-answer text never sees an answer end
• Checks that truncation did not cut the end token off long examples, and that padding with the same id is masked to -100 in labels
• Confirms generation stops on that token (eos_token_id) and sets max_new_tokens rather than a total max_length that includes the prompt
• Adds answer length and repeated-sentence rate to the evaluation, since exact match never shows this failure

2. The support lead wants to fine-tune the assistant “so it knows the new refund policy and answers in our house style”. What do you build?

What a strong answer covers:

• Splits the request: the policy is a fact that changes, so it goes in the T5.1 index and is cited; the house style is behaviour, which fine-tuning can carry
• Starts with a prompt that states the style and shows two examples, and fine-tunes only if that fails on a measured set or costs too many tokens per call
• Builds training examples whose answers follow the style but take their facts from retrieved context, so the model learns to use context rather than memorise the policy
• Plans for the next policy change: re-index, no retraining, and a golden question that checks the new window is cited

3. Your fine-tuned model’s training loss fell for two epochs. How do you decide whether it replaces the prompted model in production?

What a strong answer covers:

• Evaluates on a held-out split the model never trained on (the notebooks hold out 10%) plus the T5.1 golden questions, never on training loss
• Scores what was trained: the share of answers that lead with Yes or No, faithfulness to the retrieved policy, refusal rate on off-topic questions such as What do you think of Mars?
• Runs a general benchmark (lesson 6 of T14.1 uses ARC Easy) before and after, to catch lost general ability
• Compares against the prompted baseline on the same questions, with cost and latency per call, and reviews the regressions question by question

4. Ten percent of Larkspur’s transcripts are longer than the 512-token limit you can afford in training. What do you do with them?

What a strong answer covers:

• Measures the length distribution first; lesson 4 of T14.1’s lamini_docs rows peak at 479 tokens, well under the 2,048 the notebook allows, so its truncation never fires
• Knows the trade-off: right truncation cuts the end of the answer and its end token, left truncation cuts the question and the template header
• Prefers fixing the data (drop long greetings and signatures, split multi-turn threads into single exchanges) over silently cutting
• Logs how many rows were truncated on every run, so a change in the data shows up as a number

5. The team has one 24 GB GPU and wants to fine-tune a 7-billion-parameter model. Is that feasible, and with which settings?

What a strong answer covers:

• Does the arithmetic: Llama 2 7B has 6.74 billion parameters, 13.5 GB of weights in 16-bit before any gradients or optimiser state, so full fine-tuning does not fit
• Loads the base weights in 4 bits (about 3.4 GB) and trains LoRA adapters: at rank 16 on every linear layer that is 40.0 million trainable parameters, 0.59% of the model
• Uses a LoRA-sized learning rate; AutoTrain’s default is 3e-5, and the notebook’s 0.4 is more than 13,000 times that
• Keeps batch size small with gradient accumulation and gradient checkpointing, and merges or serves the adapter separately depending on how many variants are deployed