Enterprise security: understand the problem
- Source
- From the Staff AI System Design course
- By
- Sourav Banerjee
- Last updated
- 8 Nov 2026
Alice’s company will allow its supplier-policy workflow only if another organisation cannot read its data, revoked access stops at a defined boundary, and sensitive retrieval is auditable. A fast, fairly scheduled answer is still a failure if it crosses the wrong tenant or provider boundary. Strengthen those boundaries without replacing the platform built in T18.2–T18.6.
The customer, workload, policy ages and costs are hypothetical. The executable verifies local signed-claim fixtures, policy decisions and durable audit behaviour against the actual T18.4 retrieval code. Its public demonstration keys are not secrets; the envelope is not JWT/OIDC. No production identity provider, mTLS, private endpoint, KMS, firewall or row-level security deployment is made. Estimated integrated study: seven to nine hours; learner mastery remains unassessed.
Preserve the Actual Platform
T18.6 bounded tenant queues, compared fair service shares and guarded provider request/token/concurrency budgets. Its snapshot and ADR-005 are the before-state. Preserve shared history and stable keys, durable worker epochs, versioned ingestion, current retrieval gates, scoped immutable caches, ready replicas and unknown-outcome handling.
Before diagram. Baseline tenant predicates and budgets exist, but they are not a complete enterprise identity, audit or network control plane. Existing fixtures cannot establish real corporate identity.
Trace an Attempted Tenant Escape
Bob sends a valid Tenant B request with X-Tenant: A and a guessed citation for Alice’s supplier policy. A gateway that forwards this header as authority turns a caller claim into cross-tenant access. A model instruction such as “only answer for Tenant B” cannot repair text already retrieved from Tenant A.
A second failure is less visible. Alice was allowed yesterday; her organisation revokes her today. A long-lived token or cached “allow” may outlive that policy. A third failure concerns accountability: the system returns sensitive text but its audit write fails, leaving no durable disclosure record. Decide these consequences before adding security product names.
| Supplier-workflow consequence | Formal term | Boundary |
|---|---|---|
| Establish which account actually called | Authentication | Signed identity is verified, not inferred from a body/header |
| Decide what that account may do now | Authorisation | Current membership, action and resource policy still apply |
| Let a worker act narrowly for Alice | Delegated capability | Service audience, action, tenant, expiry and actor need explicit scope |
| Keep readable document text from becoming control instructions | Data/control separation | Authorisation/tool policy never comes from model output |
| Know which protected source IDs were disclosed | Security audit | Durable metadata record, not a raw prompt dump or an ordinary debug log |
Coverage and Learning Outcomes
| Module | Treatment | Mapped depth and evidence |
|---|---|---|
| Gateway and service communication | Taught here | Build depth: ingress trust, service audience/capability, mTLS/egress boundaries and failure tests |
| Enterprise retrieval governance | Taught here | Build depth: policy authority, freshness/epoch, current resource checks, provenance and audited disclosure |
| Identity and access control | Taught here | Build depth: signature/issuer/audience/expiry, membership, least privilege and rotation tests |
| Enterprise AI security | Taught here | Build depth: prompt/tool boundaries, scoped secret references, egress, audit failure and threat exercise |
| Multi-tenant platform design | Taught here | Build depth: shared versus isolated data/compute options, administrative paths and negative access tests |
| Networking fundamentals | Revisited from T18.2 | Design depth: locate ingress, service and provider network trust boundaries rather than repeat DNS/TLS basics |
| Distributed system fundamentals | Context only | T18.8, Build depth: authority outages/freshness matter now; failover/partition design follows |
| Cloud networking and deployment | Deferred | T18.12, Design depth: no multi-region residency/network implementation here |
Outcomes: reject forged scope; verify a bounded identity/capability; block stale/revoked context; require durable audit before disclosure; and defend isolation/egress choices. Passing fixture tests is not a security certification or proof of learner mastery.
Key Takeaways
- A tenant claim in a header such as
X-Tenant: Ais caller input, not authority. A gateway that forwards it as scope turns a guessed citation into cross-tenant access, and a model instruction cannot undo text already retrieved. - Revocation is a separate failure. A long-lived token or a cached allow can outlive the policy that removed Alice, so freshness has to be decided before any security product is chosen.
- Returning sensitive text after a failed audit write leaves no durable disclosure record. A security audit is durable metadata about protected source IDs, not a raw prompt dump or an ordinary debug log.
- Authentication, authorisation, delegated capability, data/control separation and security audit are distinct boundaries. A verified signature says who called, while current membership and resource policy say what that account may do now.