Reasoning-effort levers and cost

reasoning_effort changes one thing: how many reasoning tokens the model is willing to spend before it commits to an answer. The model, the prompt and the output format stay the same. More reasoning tokens mean more latency and more cost at the output rate, and a better answer only when the task had something to deliberate about. The notebook sweeps o4-mini at low, medium and high; the installed openai 2.54.0 also types none, minimal, xhigh and max, and the notebook says to trim the list to what your model accepts.

The sweep is six problems at three levels with TRIALS = 1: 18 calls, a few cents. The problems mix easy and hard on purpose (a train arrival time, retry backoff, the deployment puzzle from lesson 3 of T2.2, LRU eviction, a syllogism, a coin-weighing puzzle), because one question gives a coin flip, not a measurement.

Each step up the dial gets one line in the notebook's marginal table:

$ per +1pp = (cost of this level - cost of the level below) / (accuracy gain * 100)

A step with no accuracy gain reads no gain, and the notebook's rule is that every step after one is pure cost. The knee is the last step whose price per point you would pay. The usual shape, in the notebook's words, is "a step then a plateau": a real gain from low to medium, much less from medium to high, while cost and latency keep climbing.

The interactive lab prices that sweep with the notebook's own cost and marginal code, adds gpt-4o-mini from lesson 3 of T2.2 as the first rung, and scales each figure to 10,000 answers. The notebooks save no outputs, so the accuracy and reasoning-token inputs are representative: 3, 4, 5 and 5 problems right, and 250, 700 and 1,800 reasoning tokens per call. The prompt and answer tokens are counted from the notebook's six problems. With those inputs:

Three things to take from the table:

• The knee depends on what a point is worth to you. At $2 per point per 10,000 answers the lab stops at medium. Below $1.19 it stops at low, and below $0.68 the fast model wins. Going to high costs $48.40 more per 10,000 answers than medium for nothing.
• Six problems make a coarse ruler. Each problem is 17 points, so one flipped answer moves the knee. The notebook's TRIALS raises the count at linear cost; a production sweep wants 20 or more questions drawn from real traffic.
• Audit the grader before the dial. The notebook grades by substring: the expected text must appear in the answer. Its LRU problem expects c, but in a 4-entry LRU cache with accesses A B C D A E, inserting E evicts B; C goes only when B comes back. A model that answers B is marked wrong at every level and caps at 5 of 6, while a wrong C, or any reply containing the letter c such as "B is evicted", earns the point.

The notebook's last task cannot be solved by thinking: What was the total refund value processed by our EU billing service last Tuesday?, with the instruction to reply INSUFFICIENT DATA when the data is missing. Every level should reach the same reply, and the higher levels pay more reasoning tokens to get there. Flat accuracy with climbing reasoning tokens is the signature of the wrong lever; the lab's Wrong lever shape shows its bill.

The notebook's table of real fixes, as a chain:

Only the last box is an effort problem. For the Larkspur assistant, How much was my last refund? is the first box: the amount lives in the billing system, so the fix is a tool call to the account API (T7.1), and Can I get a refund after 45 days? is the last box, where effort is worth sweeping.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• reasoning_effort changes only how many reasoning tokens the model spends. Sweep it, price each step in dollars per extra accuracy point, and stop at the last step you would pay for.
• Expect a step then a plateau. Six problems make a coarse ruler, so use 20 or more real questions and audit the grader before trusting the knee.
• Flat accuracy with climbing reasoning tokens means effort is the wrong lever. Fix missing facts with retrieval or a tool, ambiguity with a clearer prompt, and mechanical tasks with a cheaper model or code.