Document Ingestion & Metadata Design: interview questions

Production questions of the kind asked in applied AI engineering interviews.

1. A customer uploads 4,000 product manuals as PDFs: two-column layouts, spec tables, scanned appendices. You have a week to make them searchable. How do you choose a parsing approach?

What a strong answer covers:

• Samples 20 to 30 pages of each layout type and compares parser output side by side before choosing
• Uses element-level parsing (or a layout model) for tables and multi-column pages, and a fast text parser for plain prose; routes by document type
• Detects scanned pages (empty or near-empty text layer) and sends only those to OCR
• Sets acceptance checks: no empty Documents, tables kept whole, page numbers that match the printed page

2. Users report that the assistant cites "page 6" when the answer is on page 7. What happened, and how do you fix it for existing data?

What a strong answer covers:

• PyPDF and PyMuPDF page metadata is 0-based; the citation code shows it raw
• Fixes it at the presentation layer at once (page + 1, or prefer page_label), so no re-index is needed
• Normalises to one 1-based page field in the schema for future ingests, and documents it
• Adds a citation test to the golden set: question, expected source and expected page

3. You run a multi-tenant RAG product. Should tenant isolation be a metadata filter in one shared index, or one index per tenant?

What a strong answer covers:

• A shared index with a tenant_id filter is cheaper to run but depends on every query path applying the filter
• Per-tenant indexes (or namespaces) make cross-tenant leaks structurally impossible and simplify deletion, at operational cost
• Whatever the choice, the filter comes from the authenticated session, never from the user's text or the model
• Tests for leaks with canary documents and a cross-tenant query suite on every release

4. Ingestion runs nightly over a document store. How do you know it is silently degrading?

What a strong answer covers:

• Tracks per-run counts by source and type: files seen, Documents produced, empty or tiny Documents, parse errors
• Alerts on distribution shifts, such as average characters per page dropping (a new scanned batch) or a type disappearing
• Remembers that many loaders fail quietly (silent_errors=True, empty text from scans, JS-rendered pages) and checks their output explicitly
• Samples parsed Documents for human or LLM review, and re-runs the golden set after big ingests

5. Your custom loader pulls 2 million support tickets from an API. It crashes at 1.4 million with an out-of-memory error. What changes?

What a strong answer covers:

• Implements lazy_load as a generator and calls it from the indexer instead of load()
• Splits, embeds and upserts in fixed-size batches so memory stays flat
• Checkpoints the last updated_at or id written, so a restart resumes instead of starting over
• Makes upserts idempotent with stable ids (ticket:{id}), so a retried batch does not create duplicates