Skip to content
Forward Deployed

Forward Deployed
Lessons and contents
Architecture decision recordsLesson 4 of 13
13 lessons
Learn

ADR-004: Stateless app replicas and immutable passage reuse

Source
From the Staff AI System Design course
By
Sourav Banerjee
Last updated
8 Nov 2026

Report an issue

Status: accepted educational architecture. Module: T18.5. Date: 2026-11-02. Supersedes: none; extends ADR-001–003 with an explicit production-persistence migration plan.

Situation and Options

Hypothetical traffic rises to 100 requests/second. A one-core 25 ms CPU path has insufficient headroom. Copying local database files would fork history, operation keys, job claims, and source authority. Query-only answer caching could leak or stale policy. Compare vertical scaling, stateless replicas, local immutable caches, and a shared distributed cache.

Decision and Migration

Choose ready stateless app replicas behind a load-balancing role, with one shared relational authority for requests/jobs/source metadata. Propose a separately operated relational service such as PostgreSQL rather than independent host-local SQLite files. Preserve schema/key scope and commit semantics; port/test claim transactions, copy/reconcile data, choose one writer at cutover, and test rollback. No managed migration or real load balancer is deployed by this artefact.

Use bounded per-replica immutable passage caches keyed by tenant/document/version/chunk. Always read current policy/version/tombstone before use. Content changes create different keys; revocation/deletion blocks reuse. TTL and LRU bound reuse, not authorisation. Cache outage may bypass to trusted storage; authority outage may not bypass to stale cached permission. Final answers, permissions, and candidate ranking are not cached here.

Evidence and Limits

Local tests reuse T18.2 App and T18.4 Knowledge against one SQLite file to verify cross-object shared history, duplicate replay, and negative access. They verify warm cache policy reads, tenant keys, version/deletion/revocation, TTL/eviction/outage, and capacity arithmetic. This is not a multi-host SQLite recommendation, a production DB migration, a benchmark, or a real autoscaler.

Capacity is modelled as the minimum app CPU, query-count DB budget, and provider budget. Four 70%-planned replicas can supply 112 RPS only under the illustrative CPU assumption. Cold caches, connection pools, provider tokens, and startup/drain time must be measured. Advanced cache stampede controls and distributed cache operations remain unimplemented.

Operational Checks and Revisit

Observe offered/admitted/completed rate, pool wait, query time, cache misses/forbidden hits, provider wait, and readiness/drain time. T18.6 adds explicit admission/fairness under fixed dependency budgets. Revisit cache scope only with a documented policy/version/model dependency contract; never remove current trusted checks for a faster graph.

Key Takeaways

  • Scale with ready stateless app replicas behind a load-balancing role and one shared relational authority, such as PostgreSQL, for requests, jobs and source metadata. Per-replica caches hold only immutable passages keyed by tenant, document, version and chunk.
  • Copying local SQLite files per host is rejected because it would fork history, operation keys, job claims and source authority. Query-only answer caching is ruled out because it could leak or serve stale policy.
  • Every cache hit still reads current policy, version and tombstone, and an authority outage may never fall back to cached permission. Cache scope reopens only with a documented policy, version and model dependency contract.

Rate your confidence and keep notes on the module page.