Staff-level platform architecture: staff interview practice
- Source
- From the Staff AI System Design course
- By
- Sourav Banerjee
- Last updated
- 8 Nov 2026
Interview 1: Defend the Platform Boundary
Prompt: Why centralise configuration but keep request handling regional?
Spoken answer: “The recurring incident is inconsistent enforcement across copied app integrations. I would centralise validation and supported contracts while customer data and execution stay inside approved regional cells. Route records live in each cell's authority store; the control plane only signs and publishes generations on opaque IDs. Current policy, audit, budgets and saved operation identity remain runtime checks. A control-plane outage freezes new cutovers. It does not stop authorised regional work, because cells keep their last-known-good generation.”
Trade-off: contract compatibility and platform on-call replace some duplicated app work.
Follow-up: the control plane is down for two hours during an incident. Can you still roll back? Yes, if the previous route is currently eligible. The cell writes a new local generation, N+2, carrying N's route, with actor and reason in its audit. On reconnect it reports N+2, and because a cell never applies an older generation, the failed N+1 cannot return. New cutovers wait for the control plane.
Interview 2: Defend the Migration Against Dual Writes
Prompt: A reviewer proposes writing every answer to old and new databases for safety.
Spoken answer: “I would ask which store is authoritative and what happens when only one write commits. Independent dual writes can fork saved answers and operation keys. I would prefer one writer, additive schema changes, and replayable projections where eventual views suffice. The pilot changes routing with an expected generation and records the old route and a history row atomically. A rollback changes future eligible work. It does not erase uncertain external calls or undo incompatible data migrations.”
Trade-off: phased compatibility takes longer than a one-time rewrite.
Follow-up: the new schema drops a column the old path still reads. Can you roll back? Not by routing. Stop before cutover and make the change additive, or design an explicit forward-repair path, because a route rollback cannot restore deleted data.
Interview 3: Defend the Cost Case
Prompt: Finance asks when the platform pays back.
Spoken answer: “I would show the inputs separately: 120 duplicated hours a month, 70 shared hours, $100 an hour, $1,000 of infrastructure and 600 migration hours. Common serving spend cancels in the comparison but stays in total cost. After migration the platform saves $4,000 a month. The overlap matters most: while cohorts move, duplicated work and platform cost both run. With six overlap months, up-front cost is $108,000, break-even is month 33 and the three-year saving is $12,000. I would validate pilot effort and claim no revenue gain from hypothetical faster delivery.”
Trade-off: centralisation may improve policy consistency even when labour savings are weak.
Follow-up: finance expects a ten-month overlap. Does the case survive? Not on labour alone. Break-even moves to month 45 and the platform costs $36,000 more over three years. I would shorten the overlap with per-cohort retirement dates, or argue from measured reductions in policy incidents, not from an invented return on investment (ROI).
Interview 4: Lead a Disagreement
Prompt: One senior engineer wants microservices everywhere; another wants only a shared library.
Spoken answer: “I would agree hard constraints first: EU data handling, current access, durable operations and bounded model capacity. Then I would compare each option on delivery, ownership, failure isolation, compatibility and cost. The library helps adoption but cannot be our sole runtime authority. Separate services help only where independent operation has value. I would choose a reversible pilot, record dissent and rejection reasons, name the decision owner and set a measurable revisit trigger.”
Trade-off: time-boxed alignment still leaves one responsible owner making the decision.
Follow-up: the pilot misses its gate by one mismatch, and the microservice advocate asks to relax the gate. What do you do? Pause the rollout and investigate the mismatch. Gates are agreed before the result; changing one afterwards needs a new recorded decision with its owner.
Interview 5: Walk Me Through the Whole Platform
Prompt: Defend the complete design, T18.2 to T18.13, in about a minute.
Spoken answer: “Alice in Tenant A submits a supplier request. An authenticated API commits the answer before reporting success, and long work runs as durable jobs with idempotent operation keys. Retrieval returns only current, authorised source versions. Stateless replicas and immutable caches scale reads; fair admission protects each tenant. Identity, audit and approved egress live in trusted services, never in the prompt. Unknown outcomes are reconciled, not blindly retried. Releases pass frozen evaluation; model routes, GPU capacity and any adapter are approved with lineage. T18.12 placed A and B in EU cells and C in a US cell, each with one fenced writer. T18.13 adds versioned manifests and a control plane that signs route generations on opaque IDs, while each cell keeps route authority. Migration is a strangler with bound shadow evidence, one owner per decision, and a 33-month break-even under illustrative inputs.”
Trade-off: every layer keeps its own guarantee, so the design has more owners and contracts than a single service, and each one must be staffed.
Follow-up: a new Tenant D must keep its data in a country the platform does not serve yet. What changes? Add a cell there with its own residency hostname, authority store and three-location quorum, and give D opaque placement in the control plane. Manifests, contracts and migration gates stay the same. The cost is a new cell's capacity reserve, on-call and witness site, and D is not onboarded until its fencing and recovery tests pass.
Capstone Defence and Verified Coverage
Run a 45-minute review: five minutes for requirements, ten for architecture and request flow, ten for outage, deletion and unknown-outcome challenges, ten for migration and economics, and ten for critique. The reviewer changes one constraint: zero data loss, lost policy authority, revoked old adapter, or double support demand. Record the changed decision, evidence needed, accountable owner and next verification step.
Solo path, when no reviewer is available: record a two-minute answer for each changed constraint, wait a day, score each recording against the rubric, then compare with the model answers.
Capstone rubric. Pass with at least four of five, and the fifth is mandatory:
- States which guarantee the changed constraint affects within the first 30 seconds.
- Names one accountable owner for the new decision, not a group.
- Names the evidence needed and how it would be observed.
- Names what stops or reverses the change, and what that reversal cannot undo.
- Never trades residency or current access for availability.
Model answers for the four changed constraints
Zero data loss. T18.12's asynchronous copies allow loss within the recovery point objective (RPO). Zero loss for acknowledged answers needs a synchronous commit to a second EU location before success, which adds cross-site latency to every write and lowers availability during a partition. Owner: the database and SRE owner. Evidence: measured commit latency and a failover test with no lost acknowledged write. The route platform does not change; it cannot create durability.
Lost policy authority. If the region's authority store loses its quorum, no current access decision is possible. The cell fails closed for work that needs one, and cutovers and rollbacks freeze because eligibility cannot be checked. Owner: the security and data owner, with the incident commander. Signal: quorum loss and policy freshness age. Restore quorum; never fall back to a stale replica for revocation.
Revoked old adapter. Rollback to any route that uses it is blocked, as in Checkpoint 3. T18.11 lineage lists every manifest pinned to that adapter. Choose another currently approved route or repair forward. Owner: model operations with each app owner. Evidence: the revocation record and the list of affected routes.
Double support demand. Support rises from 20 to 40 hours a month, so the platform costs $10,000 a month and saves $2,000. With a six-month overlap, break-even moves to month 66 and the platform costs $60,000 more over three years. ADR-012's revisit trigger fires. Owner: the platform owner with finance. Options: cut support load with self-service documentation, slow adoption, or justify the platform on measured policy-incident reduction.
The source map contains 56 modules across 18 domains and 12 stages. A map row or a green local fixture cannot establish a production deployment or distributed-systems implementation.
Remaining gaps are concrete: real identity and private routing, a deployed authority store and distributed fencing, signed route distribution, crash, power-loss and backup tests, real brokers and reconciliation, representative model evaluations, real training and checkpoint restore, provider and GPU capacity and invoices, end-to-end regional recovery, load and SLO evidence, external security review, screen-reader assessment and learner assessment. GPU kernels, distributed optimiser implementation, pretraining and machine-unlearning research remain deferred specialisations. Prioritise the first pilot's real integration risks before that optional research track.
Key Takeaways
- Centralise validation and contracts while data and execution stay in regional cells. A control-plane outage freezes new cutovers but not authorised work, and rollback still works through a local N+2 if the previous route is currently eligible.
- Independent dual writes to old and new databases can fork saved answers and operation keys. Prefer one writer, additive schema changes and replayable projections, because a route rollback cannot restore a dropped column.
- The payback case turns on the overlap. Six months gives break-even at month 33 and $12,000 saved over three years, while ten months moves break-even to month 45 and costs $36,000 more.
- The capstone's mandatory rule is never to trade residency or current access for availability. Zero data loss for acknowledged answers needs a synchronous commit to a second EU location, which adds cross-site latency to every write.