Production AI quality and releases: validate and carry forward

1. What is bound to a report? The exact bundle, dataset, evaluator and approved gate settings.
2. Why evaluate slices? A global score can hide access/refusal or tenant-specific regressions.
3. What do synthetic cost/latency values prove? Only that deterministic gate logic handles those labels, not real model performance.
4. What makes shadow costly or risky? Extra authorised inference, data handling and possible side effects unless constrained.
5. What does rollback preserve? Current safety authority, committed result identity and unknown external evidence.

The 60-second card: freeze versions; validate coverage; score slices; hard-stop known safety failure; bind gates; expose eligible sticky cohorts; observe redacted versions; change routing with CAS; preserve prior outcomes. Review after about one, seven and 21 days.

• Explain evaluation evidence versus production approval without notes in two minutes.
• Draw dataset/evaluator/gates, release pointer, runtime and telemetry boundaries.
• Defend offline, shadow, canary and human approval choices under changed risk.
• Diagnose quality/safety/cost/latency regressions with versioned evidence.
• Run/modify the case-binding, release and rollback tests; learner mastery remains unassessed.

ADR-008 and the snapshot preserve actual artefacts.

A future T18.10 compares hosted and self-hosted inference under explicit performance, policy and cost assumptions. It must use approved versioned routes, current data/tenant policy, bounded provider budgets and the inherited unknown-outcome rules, rather than treating a cheaper endpoint as an automatic fallback.

• Google SRE canarying releases: controlled partial exposure and evaluation.
• OpenTelemetry signals: metrics/logs/traces used by the proposed version-aware diagnostic design.
• NIST AI RMF: governance context, not claimed compliance.

Sources checked 2 November 2026. Cases, output strings, latencies, costs and thresholds are authored fixtures. Validation separates executed scoring/control tests from untested model/deployment behaviour.

• A report binds the exact bundle, dataset, evaluator and approved gate settings. Slices matter because a global score can hide access, refusal or tenant-specific regressions.
• Synthetic cost and latency values prove only that the deterministic gate logic handles those labels, not real model performance.
• Shadow traffic costs extra authorised inference and data handling, and it can cause side effects unless the execution path is constrained.
• Rollback preserves current safety authority, committed result identity and evidence of unknown external effects.