Interview prompt
Your GenAI FDE (Forward Deployed Engineer) task is a Natural-Language-to-SQL Analytics Assistant. Every week, executives repeat the same questions, and analysts turn each one into a query by hand. The worst outcome is not broken SQL. It is SQL that runs cleanly but answers a different business question from the one asked.
Clarify the customer problem
- Who defines each metric, and how often does a definition change?
- Is there a semantic layer already, or do we have to build one?
- Does the warehouse enforce row and column security today?
- Which warehouses and SQL dialects must we support?
- If a question is ambiguous, should the assistant ask back, decline or hand it to an analyst?
Users and workflows
| User | Workflow | Current pain | AI assist opportunity | Human approval needed? |
|---|---|---|---|---|
| Executive | ||||
| Analyst | ||||
| Data / platform engineer | ||||
| Metric owner |
Requirements
Functional
Non-functional
- Latency target
- Availability target
- Cost budget
- Security/privacy constraints
- Audit/compliance requirement
Data and integration map
| Data source | Format | Owner | Freshness | Permission model | Risk |
|---|---|---|---|---|---|
Proposed architecture
Pick a rendered diagram as your starting point and adapt it to this case. Cover each part below:
- Metric registry
- Schema retrieval
- Ambiguity check
- AST and policy validation
- Read-only execution
- Monitoring
Evaluation plan
| Metric | Good threshold | Bad threshold | Test dataset | Owner |
|---|---|---|---|---|
| Semantic correctness | Agrees with the analyst | Even one unflagged mismatch | Golden cases of question plus SQL | Analytics lead |
| Policy violations | None get through | Any that get through | Audit of query runs | Security team |
| Bytes scanned per answer | At or under baseline | More than 20% over | Telemetry from the warehouse | Data platform team |
| User trust score | At or above the launch target | Falls two periods running | Feedback captured in the product | Product team |
Failure modes
- The SQL runs but answers the wrong business question
- A schema change quietly breaks a saved template
- A join multiplies rows and inflates totals
- A query reads far more data than it needs
- The written summary disagrees with the result table
- Policy blocks leave users frustrated
Rollout plan
- Start with ten governed metrics, each with a named owner.
- Build golden cases: question, SQL and expected result.
- Re-run the golden set on every change.
- Shadow the analysts; their answer stays the source of truth.
- Open to executives once the answers match.
- Grow one domain at a time.
Weak vs strong answer
Weak: "I'd give the model the schema, let it write SQL and run whatever comes back."
Strong: "Before touching any table, I'd map the question to a governed metric. If the question is ambiguous, I'd ask rather than guess. I'd parse the SQL into an AST (abstract syntax tree), check it against policy and a scan budget, run it read-only, and prove the whole thing on ten metrics with golden cases."
Candidate scorecard
Final 2-minute spoken answer
Rehearse this out loud before you look at it. Summarising the whole design in two minutes is its own skill, and it is usually the last thing you are asked for.