Scale the application: validate and carry forward
- Source
- From the Staff AI System Design course
- By
- Sourav Banerjee
- Last updated
- 8 Nov 2026
Recall and Exit Gates
- What can be lost with a replica? Its expendable cache, not the only operation/history state.
- Why retain two authoritative checks in the hit-rate formula? A content hit does not remove trusted policy/version work.
- What is the limiting capacity? The minimum compatible app, DB, provider, and other binding budgets, using the same units.
- Why are TTL and version different? TTL bounds reuse time; the version identifies content, and live policy controls access.
- What is not measured here? Actual database throughput, managed migration, load balancing, autoscaling, and production p95.
The 60-second card: preserve one durable authority; route only to ready replicas; cache immutable scoped bytes; recheck permission; count DB pools; model the minimum resource; keep burst headroom. Review after about one, seven, and 21 days.
- Explain replica state, authority, and cache scope without notes in two minutes.
- Draw retained ingestion/workers plus the new interactive replica path.
- Defend vertical scaling, replicas, and local/distributed cache choices under changed demand.
- Diagnose CPU, DB, provider, and warmup limitations with appropriate signals.
- Run/modify cross-replica/cache/capacity tests and separate assumptions from measurements; learner mastery remains unassessed.
Decisions and the Next Pressure
ADR-004 records the explicit shared-state migration and immutable-body cache choice. The stage snapshot preserves the educational evidence.
A later T18.6 will address one tenant flooding a fixed model budget. It must preserve scoped history and retrieval, bounded cache safety, durable worker authority, and the fact that a scheduler cannot create provider capacity. This stage does not quietly implement fair admission or distributed quotas.
Primary References
- PostgreSQL explicit locking: transaction/locking semantics that must be reviewed during the proposed migration.
- Kubernetes horizontal pod autoscaling: example of a metric-driven controller, not an installed platform requirement.
- RFC 9111: HTTP cache behaviour, distinct from the application passage cache.
Sources checked 2 November 2026. All model budgets are illustrative. Tests and unverified deployment boundaries are reported in validation.
Key Takeaways
- A replica may lose its expendable cache, never the only copy of operation or history state.
- The two authoritative checks are never multiplied by the hit rate, because a content hit does not remove trusted policy or version work.
- TTL bounds how long content is reused, the version identifies the content, and live policy controls access.
- The limiting capacity is the minimum of the compatible app, database, provider and other binding budgets in the same units. Actual database throughput, load balancing, autoscaling and production p95 remain unmeasured.