STORM: multi-perspective research and writing

What you'll learn

• Explain STORM's four stages and why each one consumes the previous stage's typed output.
• Diagnose the two steps that decide article quality: persona design and outline synthesis.
• Calculate a run's model calls from its parameters before it starts, and its latency once the interviews run in parallel.
• Identify where retrieval has to plug in before the article's facts can be trusted.

Lesson 2 of T11.2 used two perspectives to argue toward a decision. STORM uses several perspectives that cooperate, to research a topic before anyone writes about it.

Larkspur's documentation team wants a help-centre article on what happens to an account when a customer cancels. A single "write an article about cancelling" prompt returns fluent, generic prose from one angle. It mentions refunds in a sentence, skips the 30-day data-retention window entirely, and no one can tell which claim came from where.

STORM (Synthesis of Topic Outlines through Retrieval and Multi-perspective question asking; Shao et al., NAACL 2024, arXiv:2402.14207) writes a long article on a topic the writer knows little about. It researches first, from several perspectives, and writes only from what the research found:

Figure 1: STORM's pipeline. Each arrow carries a typed object, not free text.

1. Perspective generation. One structured-output call proposes P distinct editor personas, each a different professional lens on the topic.
2. Simulated interviews. For each persona, a persona-editor asks one focused question at a time and an expert answers, for T turns. Both roles are the same model with different system prompts. A third call extracts the transcript's facts into a typed list.
3. Outline synthesis. An editor-in-chief call merges every persona's facts into one ordered outline of 4 to 6 sections.
4. Section drafting. A writer call drafts each section from the research notes. A last call writes the title.

The notebook runs this on the impact of remote work on urban commercial real estate, with three perspectives and two interview turns each.

Each stage hands the next one a typed object produced with structured output, and no stage parses free text:

Table 1: What travels between STORM's stages in the notebook.

Only the extracted facts travel into the outline and the drafts. The transcripts stay behind for audit, and you can trace which question surfaced which fact. Two steps decide quality, and each has a named failure:

• Echo panel. Three reworded generalists produce redundant research. If the personas overlap, tighten the perspective prompt, for example by listing the disciplines to draw from.
• Stapled outline. Outline synthesis is where P research threads become one narrative. An outline with one section per persona is a concatenation, and the synthesis prompt needs to ask for a merged structure.

A good outline reads as one article's table of contents, moving from background through each stakeholder's stake to an outlook. A stapled one reads as three reports in a row.

The notebook's expert is the model playing a knowledgeable role, with no retrieval. Its prompt asks it to label a guess as an estimate, but its "facts" are still only as reliable as the model's memory. We call this the memory expert, and it is the pattern's main weakness. The original STORM retrieves from the web at this step. The notebook swaps that out to keep the run self-contained.

Swapping the expert's plain call for a tool-using agent grounds it without changing any later stage. On the Larkspur handbook the expert would answer from a handbook retriever. For the cancellation article, each persona would surface its own facts with the chunk they came from:

Table 2: What a grounded expert would return for the cancellation article. Illustrative questions, facts from the Larkspur handbook.

Interview tip: when asked how to make STORM trustworthy, name the step. The interview's expert is the only place facts enter the pipeline, and it is where retrieval and citations go.

The cost is fixed by the parameters before the run starts: one call for the perspectives, 2T + 1 per interview (T questions, T answers, one extraction), one for the outline, one per section and one for the title. In total the run makes 1 + P × (2T + 1) + 1 + S + 1 calls:

Table 3: Model calls per STORM run, from the formula.

The notebook attaches a callback that counts every chat-model call, structured-output calls included, and compares the count with this arithmetic. Two caps keep the cost bounded when the model returns more than it was asked for: the perspective list is cut to P, and the outline to MAX_SECTIONS = 6.

Latency is a different number from cost. The interviews are independent of each other, but the notebook runs them in a Python loop, and their waits add up. Dispatching one branch per persona with Send (lesson 5 of T11.2's map-reduce) puts them in one superstep. The section drafts are independent in the same way. Counting the calls that must wait for one another:

Table 4: The longest chain of dependent calls in one run, assuming every call takes about the same time (illustrative).

The same 23 calls are made either way, and the bill does not change. The reader waits for 9 calls in a row instead of 23.

1. Perspectives: ask for three personas and name the disciplines (billing, data handling, account security) to avoid an echo panel.
2. Interviews: give the expert the handbook retriever, and keep each fact's chunk id with it in ResearchFindings.
3. Outline: ask for one merged structure, such as before you cancel, the refund, your data, closing the account, rather than one section per persona.
4. Drafts: write each section from the facts with their chunk ids, and dispatch the sections in parallel.
5. Check: compare the callback's call count with the formula, and spot-check one sentence per section against its source chunk.

Lesson 4 of T11.2 moves from fixed pipelines like this one to controllers that choose which specialist, or which whole architecture, handles a request.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• STORM researches before it writes. P personas each run a T-turn interview, their extracted facts merge into one outline, and each section is drafted from what the research found.
• Quality rests on distinct personas and an outline that merges the research threads. An echo panel repeats itself, a stapled outline is a concatenation, and a memory expert is only as reliable as the model's memory.
• Cost is fixed before the run at 1 + P × (2T + 1) + 1 + S + 1 calls, 22 to 24 in the recorded configuration. Dispatch the interviews and sections with Send to cut the chain of dependent calls without changing the bill.