Fine-tuning and model customisation: validate and carry forward
- Source
- From the Staff AI System Design course
- By
- Sourav Banerjee
- Last updated
- 8 Nov 2026
Recall and Exit Gates
- What changes first for a factual supplier-policy error? Current authorised retrieval and source freshness, not automatically model parameters.
- Why split by supplier/thread? Near-duplicate examples across splits can inflate reported generalisation.
- Is a checkpoint a deployable release? No. Recoverable training state and approved serving artefacts have different contracts.
- Can 99% schema success override one known safety failure? No. Safety gates remain independent.
- What stays unchanged after a release rollback? Saved operation identity, current access policy, audit requirements, and uncertain external outcomes.
60-second card: classify behaviour versus facts; try the smallest intervention; freeze approved data and splits; pin base/recipe; verify checkpoints; bind evaluation; approve independently; deploy through existing gateway gates; retire descendants through lineage.
Evidence and Carry Forward
Read the architecture snapshot and decision. Local content verification and learner completion are separate. T18.12 inherits approved candidate lineage and the unchanged runtime, then adds regional boundaries. No actual model training or deployment has occurred.
Review after about one, seven, and 21 days. Stop when you can explain the change in two minutes, draw it within the inherited platform, defend two alternatives, and diagnose a fault from an observable signal. For build depth modules also build, test, and debug the representative slice. These learner gates remain unassessed. Production integrations and deployment controls described here are proposals unless the validation report explicitly records an executed check.
Key Takeaways
- A factual supplier-policy error is fixed through current authorised retrieval and source freshness, not by changing model parameters automatically.
- Split by supplier or thread, because near-duplicate examples across splits inflate reported generalisation. A checkpoint is recoverable training state, not a deployable release.
- Even 99% schema success cannot override one known safety failure. After a release rollback, saved operation identity, current access policy, audit requirements and uncertain external outcomes stay unchanged.