Self-querying: natural language to metadata filter

What you'll learn

• Explain why similarity search cannot answer "more than", "after" or "one of".
• Trace a question as an LLM splits it into a semantic query and a structured filter.
• Explain why the metadata description is a prompt, and why a field's declared type matters.
• Diagnose a self-querying retriever that returns nothing, or quietly drops its filter.

Lesson 7 of T6.2 routed whole questions to a destination. Some questions need a different split: part of the question is about meaning, and part is a condition on a fact.

A Larkspur customer asks Which plans have more than 8 GB of memory? The three plan sheets (from the T6.1 lab corpus) say LC-204 has 4 GB, LC-408 16 GB and LC-816 32 GB. Similarity search ranks them by how much each sheet resembles the question:

Table 1: Similarity scores for the memory question (all-MiniLM-L6-v2, T6.1 lab corpus).

The three sheets score within 0.025 of each other, because they read alike. At k = 3 the 4 GB plan is in the context, and the model has to notice that it fails the condition. "More than 8" has a vector, and the vector cannot perform a comparison. We call this the comparison blind spot.

Notebook 08a shows the same failure on a book catalogue. Asked for books rated above 4.5, plain similarity search returns books rated 4.3, because "rated above 4.5" sits near every book description that mentions ratings.

A self-querying retriever puts an LLM in front of the retriever. The LLM splits the question into two parts, and each goes to the tool that can evaluate it:

Figure 1: Self-querying. Meaning goes to the embeddings; facts go to the metadata index.

08a's catalogue makes the division concrete. Each book's embedded text is a short thematic description, and its metadata holds seven attributes: title, author, year, genre, rating, language and country. The embedded text answers questions about themes; the metadata answers questions about facts. Here are the structured queries the LLM built:

Table 2: Structured queries. The first three rows are 08a's; the Larkspur row assumes the plan sheets carry a memory_gb field.

The constraint is removed from the semantic query and re-expressed as a filter. A pure metadata lookup leaves almost nothing to embed, and a purely semantic question produces no filter and degrades to ordinary similarity search. For Larkspur, the filter keeps LC-408 and LC-816 and drops LC-204, whatever their similarity scores.

08a exercises each comparison the filter language offers: equality on a category, a numeric greater-than, a range built from two comparisons joined by AND, and set membership built with OR.

The LLM cannot guess which fields exist. A list of attribute descriptions, one per field with a name, a type and a sentence, is written into the query-construction prompt. Two details carry weight:

• The type permits comparisons. Declared as a float, rating allows greater-than; declared as a string, the model can only test equality.
• The description is a prompt, not documentation. "The rating of the book (1-5 scale)" tells the model what values are plausible.

A field that is not described cannot be filtered on. A field that was never written at ingestion cannot be filtered on either, which is lesson 5 of T6.1's point again: facets come from ingestion.

08a also lets the question set the result count. With the limit option on, "two books" becomes a limit of 2; without it, the phrase is silently ignored and the default k applies.

The query constructor is an LLM, so its output varies between runs. 08a asked the same question four times and recorded what came back:

Table 3: Four runs of one question through the query constructor (08a).

Two failures hide in that table. The string value crashes the call, and 08a fixes it by coercing numeric filter values before they reach the store. The missing filter is worse, because nothing fails: the retriever quietly falls back to similarity search. We call this the dropped filter. 08a's helper prints the filter the LLM produced beside every result, so a dropped filter is visible.

Pitfall: an empty result is not "nothing matched". A filter on "annual" when the metadata says "Annual", or a field the schema never declared, returns zero chunks with no error. Log the structured query for every call, and treat an empty filtered result as a reason to retry without the filter or to ask the user.

08a also passes the store's query translator explicitly, because the automatic detection broke when one vector store integration moved to a different package. That is a version detail; the lasting point is that the translator renders the structured query into the store's own filter syntax.

1. Write facets at ingestion: plan name, memory, vCPUs and traffic on the plan sheets; section, version and date on policy chunks.
2. Describe each field with a type and one sentence, numbers as numbers.
3. Coerce and log every structured query, and alert on calls where a question with a number produced no filter.
4. Route here only questions with a condition on a fact (Figure 2 in lesson 1 of T6.2); everything else keeps plain or transformed retrieval.

This closes the query side of retrieval. T6.3 turns to the other side: what to do with the chunks once they come back.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• Embeddings compare meaning, not numbers (the comparison blind spot): the three Larkspur plan sheets score within 0.025 of each other for "more than 8 GB", and the 4 GB plan makes the top 3.
• A self-querying retriever has an LLM split the question into a semantic query and a metadata filter; the schema of field names, types and descriptions is part of its prompt.
• The constructor's output varies: coerce numeric values, log the structured query, and watch for the dropped filter, which falls back to similarity search without an error.