Simple PDF and CSV RAG in LlamaIndex

What you'll learn

• Map LangChain's RAG pieces onto LlamaIndex's Documents, Nodes, retrievers and query engines.
• Explain why the same chunk size gives different chunks in the two frameworks.
• Diagnose an ingestion pipeline that crashes on its cleaner and leaves its vector store empty.
• Trace a question through a query engine's three stages, and read what it returns.

T5.1 to T6.5 built every technique in LangChain. The ideas are framework-independent, and rebuilding them in LlamaIndex is the test: if a port behaves differently, either the framework does something different or the idea was only half understood.

LlamaIndex has two text objects where LangChain has one. A Document is what a reader loads: the directory reader turns 01a's climate-change PDF into 33 Documents, one per page, and a paged CSV reader turns 01b's 100-row customer file into 100 Documents, one per row. A Node is what the index stores: a piece of a Document with its own id, text, metadata, an optional embedding, and relationships. Every node the sentence splitter produces carries three relationships: its source Document, the previous node and the next node. The neighbour links let a query engine fetch the text around a hit later (lesson 4 of T6.6).

The concepts map onto the LangChain pieces from T5.1 to T6.3:

Table 1: LangChain to LlamaIndex.

Two differences change the chunks you get from the same settings.

Chunk size counts tokens. 01a sets a chunk size of 200 with 50 of overlap and says so in a comment: LangChain's splitter measures characters by default, LlamaIndex's measures tokens. 200 tokens is several times longer than 200 characters, so a "200" ported from a LangChain notebook is not the same split. We call this the unit swap.

Metadata counts against the chunk and goes into the embedding. The sentence splitter builds the metadata string a node will carry, measures it in tokens and subtracts it from the chunk size before splitting. The PDF reader attaches seven fields (page label, file name, path, type, size and two dates) and excludes all but the page label and file path from the embedding and LLM text:

Table 2: Metadata eats the chunk (measured on the checkout used for this page).

The absolute file path is part of every vector. Exclude fields from the embedding and LLM text on the Documents, before splitting, when they should be stored but not embedded.

Splitting never crosses a Document boundary. Notebook 02a uses the splitter's defaults, 1,024 tokens with 200 overlap, and every PDF page fits in one chunk, so the 33 pages become exactly 33 nodes. The page, not the chunk size, decides what a chunk is. On the Larkspur handbook the two sections load as two Documents; lesson 4 of T6.6 splits them into one node per sentence, 13 for refunds and 4 for exports.

01a creates a FAISS index, passes it to an ingestion pipeline, then builds a vector store index from the nodes. With the installed llama-index-core 0.14.25 two things go wrong:

Table 3: Two ingestion bugs in 01a, repeated in every T6.6 notebook.

Run on the notebook's PDF with a local embedding model in place of OpenAI's, the fixed pipeline makes 123 embedding calls, one per node, the index makes none, and FAISS ends with 123 vectors. Rebuilding an index from the FAISS store alone fails with "Cannot initialize from a vector store that does not store text", because FAISS keeps vectors and no text.

Pitfall: the empty store. A pipeline given a vector store that it never fills raises nothing. Count the vectors in the store after ingestion, as lesson 1 of T6.4 counted vectors against chunks.

A query engine turns a question into an answer in three stages. index.as_query_engine(similarity_top_k=2) returns one, and each query runs:

Figure 1: A query engine's three stages.

• The retriever embeds the question and returns the nearest nodes with their scores. The default is 2.
• The postprocessors run in list order. 01a passes none; lessons 3 and 4 of T6.6 add a reranker and a window replacement here.
• The synthesizer puts the nodes' text into a prompt for the configured LLM. The default mode, compact, packs the nodes into as few prompts as the context window allows, then refines the answer across them.

The response holds the answer and the source nodes it was built from, with their scores. 01a's evaluation reads both: the answer goes to correctness and faithfulness metrics, the source texts become the retrieval context for DeepEval's contextual relevancy metric. Logging the source nodes on every production query gives the same view without an evaluation run.

The engine can also run the retriever and postprocessors without the LLM, which is how lesson 4 of T6.6's lab was checked. Building the engine still needs an LLM: it resolves the configured model when created, and with none set and no OpenAI key it raises before any retrieval. Set a mock LLM for tests, or take a plain retriever from the index when you only need nodes.

01b reads each CSV row as one Document whose text is one "column: value" line per field: index, customer id, first name, last name, company and so on. It skips splitting, embeds each row, and asks which company does sheryl Baxter work for? with two results. The answer is in row 1 (Rasmussen Group), and the row's own text names her, so one retrieved row is enough.

A question about many rows, such as how many customers are in Chile?, gets the same two rows, and no prompt can count the other 98. That is lesson 4 of T6.4's sample average again: row-per-Document RAG answers lookups, and counts, sums and filters belong to a query over the table.

1. Convert the chunk size: Larkspur's 300-character split is about 75 tokens at a rough four characters per token, not 300; measure it with the splitter's own tokeniser.
2. Exclude file paths from embedded and LLM metadata before splitting.
3. Embed in the pipeline and count vectors in the store afterwards.
4. Log source nodes for every answer, as lesson 2 of T6.4 kept its context.

Lesson 2 of T6.6 ports T6.1's hybrid retrieval, where LlamaIndex's defaults differ most.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• A Document is what a reader loads; a Node is what the index stores, with an id, metadata and neighbour links. Splitting never crosses a Document boundary.
• The sentence splitter counts tokens, not characters (the unit swap), and subtracts embedded metadata from the chunk size: 38 tokens of page label and path leave 162 of 200 for text.
• A query engine retrieves, postprocesses and synthesises; log its source nodes. Put the embedding model in the ingestion pipeline, or the vector store you passed stays empty.