Accuracy plateaus; the bill keeps climbing.
The six-problem sweep from lesson 2.4, priced with the notebook’s own cost formula. Pick a sweep shape, or set how many problems each setting got right and how many hidden reasoning tokens it spent, then set what one accuracy point is worth to you. The lab picks the setting past which you pay for thinking that no longer changes the answer.
- The knee
- Accuracy there
- Per 10,000 answers
- Saved against high
The sweep, one row per setting
Six problems per setting
| Setting | Correct of 6 | Reasoning tokens | Hidden share of output | Sweep cost | Per 10,000 answers |
|---|
Prompt tokens are the six problem prompts counted with o200k_base (27 to 38 each); the API adds a few framing tokens. Visible answers are the expected answers’ tokens (1 to 3). Prices are the notebook’s PRICING constants in USD per million tokens, input and output: o4-mini 1.10 and 4.40, gpt-4o-mini 0.15 and 0.60. The notebook tells you to replace them with today’s prices.
What each step up buys
The notebook’s marginal table
| Step | +accuracy | +cost, sweep | $ per +1pp, sweep | $ per +1pp, 10,000 answers |
|---|
The first three columns are what 04a_Reasoning_Effort_Levers.ipynb prints, with the fast model from lesson 2.3 added as the first rung. The last column scales the same number to 10,000 answers. The knee is the highest setting whose gain over the setting below it costs at most your threshold per point.