Skip to content
Forward Deployed

Forward Deployed
Lessons and contents
Architecture decision recordsLesson 10 of 13
13 lessons
Learn

ADR-010: Governed model customisation

Source
From the Staff AI System Design course
By
Sourav Banerjee
Last updated
8 Nov 2026

Report an issue

Status: accepted educational architecture, not production approval. Module: T18.11. Date: 2026-11-02. Supersedes: none; prior runtime invariants remain in force.

Situation

Read T18.11 and its labelled workload assumptions. The prior accepted architecture is v0.9-stage-09. The existing supplier workflow must carry its access, operation, budget, and release guarantees into this change.

Options Considered

Option Benefit Cost/risk Decision
Prompt and output validation Fast iteration without changing weights Context cost; semantic failures remain Baseline first
Authorised retrieval Current facts and provenance Index/policy freshness and token budget Retain
Tenant adapter trial Learn stable behaviour Dataset governance, evaluation, serving compatibility and retirement Bounded challenger
Full tuning Broader parameter changes More training state and operating cost Defer without evidence

Decision

Accept immutable data-to-model lineage and an independently approved adapter trial. Add approved dataset snapshots, Fine-Tuning Jobs, experiment/checkpoint records, Evaluation Service and Model Registry. Training completion creates a candidate, never a live route. Preserve current identity/resource policy, audit, egress, versioned retrieval/cache, durable operation identity, unknown-outcome handling, fair budgets, and T18.9/9 release/serving contracts.

Guarantees and Non-Guarantees

Immutable dataset identities, disjoint supplier-group splits, scoped job keys, checkpoint ordering, report binding, independent safety gates, trusted approval roles and generation checks are exercised locally. The model lifecycle fixture is in memory and receives trusted metadata/counts. It does not train, restore checkpoints, attest artefacts, enforce production identity, deploy a model, or compose the prior release/gateway fixtures. The proposed HTTP contract requires durable acceptance; the local library does not implement that durability.

Consequences and Operational Checks

See failure experiments and metrics. Preserve previous approved serving until every new eligibility gate passes. Retain protected lineage/operation evidence during rollback; never roll back current permissions, tombstones, or unknown external effects.

Revisit Trigger

Changed behaviour requirements, inadequate adapter quality, revoked data purpose, changed base/tokeniser, or measured training/serving cost invalidates the trial assumptions. Distributed training remains awareness depth.

Key Takeaways

  • Data-to-model lineage is immutable, and a tenant adapter is an independently approved, bounded challenger to prompt validation and authorised retrieval. Training completion creates a candidate in the Model Registry, never a live route.
  • Full tuning is deferred without evidence because it brings more training state and operating cost. The adapter trial carries its own cost in dataset governance, evaluation, serving compatibility and retirement.
  • Previous approved serving stays until every new eligibility gate passes, and rollback never touches current permissions, tombstones or unknown external effects. Changed behaviour requirements, poor adapter quality, revoked data purpose, a changed base model or tokeniser, or measured cost reopens it.

Rate your confidence and keep notes on the module page.