Reasoning-Effort Budgeting: When Thinking Tokens Become a Finance Line Item

The first time your finance team asks why a single user racked up a fifty-cent answer to a one-tenth-of-a-cent question, the call will not be about the model. It will be about the line on the invoice that did not exist twelve months ago: reasoning tokens. They look like output tokens on the bill, they bill at output-token rates on most providers, and they have no natural ceiling. A query that would have produced a four-hundred-token reply on a non-reasoning model can quietly burn eight thousand internal thinking tokens to get there — and the only person who notices is the one reconciling the spend.

For most of the API era, "tokens used" was an honest number. You sent a prompt in, you got a response out, and the bill was a clean function of both. Reasoning models broke that intuition. The model now generates a hidden, billable, internally-only-visible chain of thought before it emits the answer the caller will read, and the size of that chain depends on the model's own assessment of how hard the question was. The user-visible output may be a single sentence. The bill may be for ten pages.

That mismatch is what makes "should this query think" the new cost-control surface. A year ago the per-task decision was about quality — does this prompt need a smarter model. Today it is also about spend, latency, and a budget envelope that finance can ask hard questions about. The teams that treat thinking as a free upgrade are the ones whose monthly invoice doubles in a quarter without an obvious cause; the teams that treat it as a knob with a measured yield curve are the ones whose CFO stops calling.

Reasoning Tokens Are Not Output Tokens, Even When They Bill That Way

The plumbing here matters more than it looks. Anthropic's extended thinking on Claude, OpenAI's reasoning models, and most third-party reasoning APIs all charge for the model's internal reasoning trace at the output-token rate. The trace itself is not returned to the caller — only a summary, or nothing at all — but it counts against max_tokens and against the bill. A 500-visible-token answer can total 2,000 to 3,000 charged tokens once thinking is included; long-form analysis routinely lands at 8,000 to 16,000. Practitioners report a 4-to-5x cost overhead in workloads as ordinary as code review.

The ceiling, if you set one, is also not where you might expect it. On Claude, budget_tokens is the soft target for thinking, and max_tokens is the hard cap on thinking-plus-text combined. The two interact. Set max_tokens too low and the model burns its thinking budget and runs out of space for the answer the user actually reads. Set budget_tokens aggressively and you cap the spend, but you also cap the model's ability to recover from its own dead ends mid-reasoning. The default is not "no thinking." On adaptive-thinking-only models like the newer Opus tier, you cannot opt out by passing the old enable-thinking shape — it returns a 400. Thinking is the floor, not the ceiling.

This is the part finance does not love: the bill is non-deterministic in a way the old bill was not. The same prompt, run twice, can think for very different lengths. A prompt that thought for 2,000 tokens last week can think for 12,000 this week because the model's internal heuristic for "this is hard" shifted with a routine update. Your unit economics now depend on a number you do not fully control, and your dashboards have to surface it as its own thing or you will not see it move.

Thinking Yield Is the Eval Metric You Probably Don't Have Yet

The benchmark question has changed. "Does the new model with thinking enabled score higher" is the wrong frame. The right frame is thinking yield — quality lift per reasoning token, not quality lift in absolute terms. Without that metric you cannot distinguish two very different worlds: thinking that cleanly pays for itself on hard tasks, and thinking that adds 30 seconds of latency and an 8x spend hike for a one-percent accuracy gain.

The research community has caught on. Recent work on reasoning efficiency has consistently shown that most thinking models sit below the efficiency frontier — their reasoning tokens do not produce proportional quality gains, and they overthink simple problems by burning tokens on questions that would be solved correctly without any chain-of-thought at all. The token economy of reasoning models has its own scaling law, and most production deployments are unknowingly operating in the flat part of the curve.

Building the eval is straightforward in principle. For each task class — extraction, classification, summarization, multi-hop reasoning, code repair — measure two numbers across a representative sample: accuracy with thinking on (at one or more budget levels), and the median reasoning-token count to reach that accuracy. Subtract the no-thinking baseline. The cost-per-accuracy-point gain is your yield. Tasks with negative or near-zero yield should default to no-thinking or routed to a non-reasoning model entirely. Tasks with steep yield should be allowed to think, possibly at the highest tier.

What this exposes is uncomfortable: a meaningful fraction of agent steps — tool-arg validation, simple JSON extraction, "did this string contain X" — light up reasoning by accident, generate three to seven times the tokens of a non-reasoning model, and produce identical answers. Tools that pass agent traces through a reasoning model uniformly are quietly subsidizing this overhead on every step.

Routing Is the Cheapest Optimization You're Probably Not Doing

The single highest-ROI architectural pattern is a router that decides whether thinking is worth it before the reasoning model is invoked. A small, fast classifier — even a regex-and-rules layer in front of a tiny model — can label incoming queries by predicted difficulty, and only the high-difficulty branch routes to the expensive reasoner. The research community has shown routing can match larger-model accuracy at roughly two-thirds the inference compute, and that's with off-the-shelf classifiers; production deployments with domain-specific signals usually do better.

The pattern that holds up across teams looks like this. A common enterprise distribution is roughly 70 percent of queries to a budget non-reasoning model, 20 percent to a mid-tier model with light thinking, and 10 percent to the heaviest reasoner. The exact numbers vary by domain, but the shape is consistent: the long tail of trivial queries should never see a reasoning model, and only a minority of traffic justifies the full thinking budget. Skipping the router and sending everything to the most capable reasoner is the equivalent of using a 200-megawatt data center to render a CSV.

The classifier itself does not need to be sophisticated. Length-based heuristics, intent-detection on the first token, and a simple difficulty regressor catch most of the easy wins. The hard part is not the model — it is the discipline to actually wire the router in front of the reasoner and resist the temptation to bypass it for "important" requests, which is how the router gets neutered into a no-op within a quarter.

Budget Caps Are Both a Technical and an Organizational Control

A budget_tokens cap is the brake. Without one, the model has no incentive — and increasingly no architectural inclination — to short-circuit its own thinking. The minimum is 1,024 on the major providers; the maximum is whatever you can afford to lose to a single runaway request. Set it too high and a malformed prompt or an adversarial input can chew through twenty dollars of compute on a single call. Set it too low and your hardest queries get truncated mid-thought and produce worse answers than no-thinking baselines.

The reasonable default is per-task, not per-call. A summarization endpoint that handles paragraph-length inputs probably never needs more than 2,000 thinking tokens; a multi-step planning agent might justify 16,000. Treat the budget like a database connection-pool size: it is a resource shape, you tune it for the workload, and you alert when calls hit the cap, because that is a signal something is wrong upstream — either the prompt is malformed, the input is adversarial, or the task is genuinely harder than you sized for.

The organizational control is harder. Without a budget governance discipline, every product team gets to set their own thinking budgets, and the people writing the prompts are not the people reading the invoice. The pattern that works is platform-owned defaults — the LLM gateway sets a budget_tokens based on the route's task class, with explicit override required to exceed it. Overrides get logged and reviewed monthly. The teams that ship without this layer end up with thirty different budget choices across thirty endpoints, half of them set by whoever was on call when the latency complaint came in.

FinOps Integration: Reasoning Tokens Need Their Own Dashboard

If reasoning-token spend is aggregated into "output tokens" on your dashboards, you will not see the cases where the model is thinking itself into an expensive answer to a cheap question. The aggregation hides exactly the failure mode you most need to surface. The solution is unglamorous: separate counters, separate alerting, separate per-tenant attribution.

The minimum viable FinOps stack for reasoning workloads has four pieces. First, per-request logging that records input tokens, visible output tokens, thinking tokens as a separate field, and the model and budget configuration. Second, attribution metadata — user, team, feature, route — so you can slice spend by something more meaningful than "the API." Third, a real-time dashboard with thinking-token spend as its own panel, not buried in output. Fourth, threshold alerts on per-user, per-feature, and per-route thinking-token rates, because the bill never blows up uniformly — it blows up on a single feature shipped on a Friday.

The privacy and storage implications are not trivial. Per-request logs that capture full thinking-mode metadata are heavier than the old "just log the input and output" approach, and the thinking traces themselves, where providers expose them, can leak signal about your prompts and your data. Most teams should log token counts and configuration without storing the raw thinking traces, and apply the usual retention discipline to the rest.

The thing this dashboard catches that no general LLM cost dashboard catches is the thinking-yield drift: the case where last week's prompt cost 2,000 reasoning tokens for the same task, this week it costs 6,000, and the accuracy did not move. That is a model-update signal, a prompt regression, or a traffic-shift signal — but it is invisible without thinking-tokens-as-a-line-item, because the absolute spend on output tokens may not move noticeably until the cumulative drift adds up across a quarter.

What "Thinking Is Free" Costs You

The default mental model — "thinking just makes the model better, turn it on for everything that matters" — is the most expensive default in the current LLM stack. The cases where it bites hardest are the ones that look benign: a JSON-extraction endpoint with thinking enabled because the original prompt designer thought it might help, a customer-support agent that thinks for 12,000 tokens before saying "I'll connect you with a human," a document-parsing pipeline where the reasoning model gets stuck verbose-listing the structure of every clause it parses. None of these show up as obvious problems in product metrics. All of them show up as line items finance will eventually ask about.

The discipline to develop is not "stop thinking." It is to treat reasoning effort as a tunable resource — measured in yield, governed by budgets, routed by difficulty, and reported as its own line on the dashboard. The teams that have this discipline get the quality benefits of reasoning models on the queries that need them, and the cost profile of non-reasoning models on the queries that don't. The teams that don't get a finance team that learns the word "tokens" the hard way.

The forward-looking version of this is straightforward: as models continue to specialize the reasoning surface — adaptive thinking, separately-priced reasoning rates, structured reasoning summaries — the per-task accounting will only get more granular. The teams who built thinking-yield evals and per-route budgets in 2026 are the ones whose unit economics will still pencil out in 2027. The ones who shipped on the default budget on every endpoint are going to spend the year scrambling to retrofit governance onto a system that already has it baked into the wrong layer.