LLM workflow apps: review analyst, IT support desk, tailored writer, quiz generator

What you'll learn

• Explain the shape every workflow app shares: typed LLM steps, then checks and rendering in code.
• Compare four applications of that shape by what the model fills and what code verifies.
• Diagnose the planted failures each check catches: a wrong route, an invented figure, an unsupported quote.
• Choose between one structured call and a chain of calls by measuring both on the same inputs.

Lesson 4 of T8.3 compared a fixed SQL workflow with an agent and found the workflow cheaper, more predictable and easier to test. This lesson applies the fixed workflow to business text: reviews, support tickets, a fact sheet, a PDF.

Larkspur's support queue gets tickets in several languages. A model can read each one, sort it and draft a reply, and most drafts are good. The bad ones read just as well. A draft that tells a customer the refund window is 14 days, when the handbook says 30, looks exactly like a correct draft until someone checks the number.

So the model's output needs a shape that code can inspect, and the decisions with a right answer need to move out of the model.

A workflow app runs a fixed sequence of LLM steps over business text, with no tool loop. The model fills typed fields, and code does everything that has a right answer: routing, checking and assembling the final document. Anything that fails a check goes to a person instead of the customer:

Figure 1: The shared shape. Only the first box is a model call.

Four notebooks apply that shape to four jobs:

• Review analyst. The model returns one typed analysis per review, for six reviews in one batch, then writes one email per review. Code picks the email writer by a dictionary lookup on the sentiment field: a thank-you for positive and neutral reviews, an apology for negative ones. Code checks the sentiment against hand labels, and each email for "Dear Customer", the "Service Agent" sign-off, an apology only when the review is negative, and at most 140 words (the prompt asks for under 120).
• IT support desk. Design A makes one structured call per ticket that returns six fields. Design B chains four calls: detect the language, translate to English, answer, translate back. Both run on the same ten tickets in nine languages. Code checks the detected language, that each reply differs from its English version, that no prompt scaffolding leaks into the text, and the calls and wall time.
• Audience-tailored writer. Three advert prompts, then three audience reports, each set run in parallel. Code checks the headings, a two-column table limited to five features, a 60-word limit, and that every number in the output appears in the source fact sheet.
• Quiz generator. The model summarises a PDF on prompt engineering, then writes questions from the material and the summary. Code checks for five questions with four distinct options and a valid answer index, checks that every evidence quote appears in the cleaned PDF text, and renders the study sheet from a template.

Five rules for the model's output

The four apps differ in content and agree on these rules:

• Ask for a closed type. A sentiment that can only be positive, negative or neutral gives code a value to branch on. An answer index points at an option instead of restating it, so the answer cannot drift from the options.
• Route in code when the signal is already structured. The review analyst picks the email writer with a dictionary and no extra model call. When the input is free text, as in its instruction router, classify into a closed set of intents and keep an other intent for requests the tool does not handle.
• Check every output with code that cannot be talked round. String and number checks are cheap enough to run on every call.
• Render the final document in code. Asking the model to merge a checked summary and checked questions into a report would let it reword them after the checks. Call that post-check rewording. A template cannot do it.
• Measure the design choice on the same inputs before picking it, as the support desk does below.

Planted failures the checks catch

Each notebook plants at least one bad output to prove its check works. The first three rows below were rerun from the notebooks' own check functions:

Table 1: One planted failure per check. Rows 1 to 3 come from running the notebooks' check functions in a scratch copy.

The second row is an invented figure, a plausible number with no source. A schema checks the table's shape and cannot know the number is false. Only the comparison with the source catches it.

The support desk is the one app that measures two designs against each other. Both start from the same ticket:

Figure 2: The two designs. Design A returns all six fields at once, and Design B makes one call per step.

The call count follows from the shapes:

Table 2: Model calls per design. Wall time depends on the provider and is measured in the notebook.

Design A makes a quarter of the calls. Design B has a second weakness: its steps exchange free text, and a step can leak a code fence or a line such as "The customer message in English is:" into the customer's reply. A typed field leaves no room for that. The notebook measures language accuracy, leaks, calls and wall time on the same tickets before choosing.

Here is the same shape on Larkspur's tickets:

1. One structured call per ticket returns the language, a category (refund, outage, sizing) and a draft reply.
2. Code routes outage tickets to the incident template.
3. Code checks that every number in a reply appears in the handbook, such as the 30-day refund window and the 99.95% availability target. The 14-day draft from the opening fails here.
4. A person sees any reply that fails a check before the customer does.
5. Actions go through the gate. A reply that issues a refund is an action. It goes through the approval gate from lesson 1 of T8.3, which binds the approval to the previewed action.

These apps are workflows in the sense of lesson 4 of T8.3: the same steps every time, cheap and testable. Choose an agent only when the next step depends on what the model finds.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• In a workflow app the model fills typed fields and code does everything that has a right answer. Ask for closed types, such as a three-value sentiment, so code can branch on them without another LLM call.
• Check every output with code that cannot be talked round, such as confirming each number appears in the source. Render the final document from a template so nothing is reworded after the checks.
• Measure a design choice on the same inputs before picking it. Send actions such as issuing a refund through an approval gate, and choose an agent only when the next step depends on what the model finds.