Enterprise knowledge: observe and experiment

The offline walkthrough follows ingestion through cited context. Choose current authorised text, a source deletion, or a permission revocation. The full static path remains below without scripting.

Sequence diagram. The final gate is a local current-policy read. A concurrent upstream permission change not yet observed is outside the fixture’s guarantee.

Version 1 says “Supplier delivery date requires written confirmation.” A second paragraph covers overdue deliveries. A separate restricted document mentions secret discounts and is readable only by manager; Tenant B has its own agreement. Querying “delivery confirmation” as Alice returns the allowed policy passage, with citation supplier-policy:v1:c0.

Version 2 replaces the policy with a portal-confirmation rule. Publication deletes the old passages and changes the current pointer atomically in the local store. A prior version-1 hit is rejected during context assembly. If a tombstone at version 3 arrives, a late version-2 upsert cannot restore search visibility. If Alice is removed from the readers list between search and context assembly, the current recheck removes the hit even though it was previously relevant.

The test fixture verifies retrieval and citation construction, not the factual correctness of a hosted generated answer. The application should separate quoted evidence from instructions, cap context, preserve citation IDs, and refuse unsupported policy claims. Document text may contain hostile instructions; being readable does not make it trusted control-plane input. Tool allowlists and ownership checks still run outside the model.

Assume 1,000 documents, 10 chunks/document, and 500 tokens/chunk. That is 10,000 chunks and five million chunk tokens before overlap. If a hypothetical embedding had 768 float32 dimensions, raw vectors alone would use 10,000 × 768 × 4 = 30,720,000 bytes, about 29.3 MiB. Index overhead, text, metadata, replicas, and old generations add storage; this is not an actual service footprint or a recommended model dimension.

At 200 changed documents/hour, 10 chunks/document yields 2,000 changed chunks/hour. At an assumed 0.5 seconds of embedding-service work/chunk, the batch requires 1,000 worker-seconds/hour. One continuously available worker has 3,600 seconds/hour in the idealised arithmetic, but quotas, parsing, bursts, retries, and batching change real capacity. The lexical fixture makes no embedding calls and proves no embedding throughput.

For a serial text-extraction capacity of 20 seconds/document inherited from T18.3, 200 changes/hour require 4,000 worker-seconds/hour before indexing. That exceeds one worker’s hour. Three independent workers have nominal headroom only if those external/storage limits permit it. Evaluate the bottleneck of each pipeline stage, not just the final model call.

Open the policy/version experiment. Two controls vary identity and evidence condition. The single illustrative candidate is version 2 of the supplier policy; the scenario describes a trusted gate, not a search latency model.

“No model call authorised” in this experiment means no call for this grounded-answer path because no evidence remains; it does not disable the separate general drafting API. Real mixed-result retrieval may keep other permitted passages. The model is deterministic, uses one candidate, and has no network or hidden relevance score.

Run this stage's Python lab in your terminal to see each idea in the stage hold. It needs Python 3.10 or later and nothing else: the standard library and SQLite only.

From the root of your local clone of the repository, run:

python3 10_AI_System_Design/stages/03-enterprise-knowledge/labs/test_knowledge.py -v

-v prints one line per test, and each test name states one behaviour. Read a test, change one value in the lab code, predict which test fails, and run it again.

knowledge.py uses a real SQLite FTS5 index and immutable/current metadata tables. Tests cover relevant scoped results, duplicate publication, content replacement, tombstone ordering, revocation between search/context, forged/stale hits, orphan publication failure, immutable-version conflict, and query syntax. A pipeline test also uses T18.3’s durable job store to claim and complete an ingestion event. Python must include SQLite FTS5 and JSON functions; a missing extension is a prerequisite failure, not permission to silently substitute an untested search engine.

For build depth practice, add an allowed policy paraphrase and an exact supplier code. Measure retrieval hits against a small labelled set before changing ranking. Compute recall@k as relevant retrieved passages divided by all labelled relevant passages for each query, and distinguish it from precision@k. The local test has a tiny relevance expectation; it is not a production quality benchmark. For explain depth object lifecycle, explain which blob can be orphaned and why pointer rollback must preserve the previous readable version.

A deletion at version 3 commits, then a delayed version-2 content event arrives. Which state should be visible?

• Search returns ranked candidates, then the context builder rechecks current version, tombstone and readers before any text leaves trusted code. Revoked, deleted or stale evidence is removed, and no remaining evidence means no grounded answer.
• After a tombstone at version 3 commits, a late version-2 upsert may leave an orphan blob but cannot restore the visible pointer or searchable passages.
• At 1,000 documents, 10 chunks each and 500 tokens a chunk, the corpus holds 10,000 chunks and five million chunk tokens. Hypothetical 768-dimension float32 vectors would need about 29.3 MiB raw, before index overhead and replicas.
• Text extraction at 20 seconds a document for 200 changes an hour needs 4,000 worker-seconds an hour, more than one worker's 3,600. Find the bottleneck in each pipeline stage, not only the final model call.