Who Pays for the Tokens? Chargeback Design for the Internal LLM Platform

Every internal LLM platform goes through the same arc. In month one, inference is free: the platform team eats the bill, product teams experiment wildly, and everyone celebrates adoption curves. By month six, the bill has grown 10x, finance is asking pointed questions, and the platform team discovers that three teams account for 80% of spend — one of them running a nightly batch job nobody remembers approving. The instinctive response is to install a meter and start charging. That instinct, applied naively, is how you kill your platform.

Here's the uncomfortable truth: per-token chargeback punishes exactly the behavior you built the platform to encourage. The team prototyping an agent that might transform your support workflow burns tokens like a furnace — agentic workloads consume 5 to 30 times more tokens per task than a simple chat completion. Bill them list price from day one and they stop prototyping. Meanwhile, the team running a mature, optimized feature pays pennies and looks virtuous. You've built a pricing system that taxes learning and rewards stagnation.

The average enterprise AI budget has grown from roughly $1.2 million a year in 2024 to around $7 million in 2026, and some large companies now report monthly inference bills in the tens of millions. At that scale, "the platform team absorbs it" stops being an answer. Someone has to pay for the tokens. The design question is who, when, and — most importantly — in what unit.

Free Inference Is a Capacity Auction Won by the Loudest Team

Start with why "free" fails, because it fails in a subtler way than most platform teams expect. The obvious failure is unbounded spend. The less obvious failure is misallocation.

When inference costs nothing at the point of use, capacity gets allocated by organizational politics rather than value. The team with the best relationship with the platform lead gets its rate limits raised. The VP who escalates loudest gets the reserved capacity. The quiet team building something genuinely valuable queues behind a vanity chatbot that a louder org shipped to impress leadership. Every shared resource without pricing degenerates into this — it's the tragedy of the commons with a GPU attached.

Free inference also destroys the information layer. Nobody can answer "is this feature worth what it costs?" because nobody knows what it costs. Retrieval pipelines re-embed unchanged documents daily. Agents retry failed tool calls with full-context resends. Debug environments hammer production models. None of this is malicious; it's just invisible. Waste that would be a P1 incident in any metered system persists for quarters because no one is looking.

So doing nothing is not an option. But the jump from "free" to "billed" has an intermediate rung that most organizations skip at their peril.

The Showback-First Ladder

The mature pattern borrowed from cloud FinOps — and now formalized in the FinOps Foundation's AI working groups — is a three-rung ladder: showback, then budgets, then chargeback. Each rung builds the muscle the next one requires.

Rung one: showback. Attribute every request to a team, project, and feature, translate tokens into dollars, and publish the numbers where everyone can see them. No consequences, no bills — just visibility.

This sounds toothless, but it does two critical things. First, it forces you to build the attribution plumbing: metadata tags on every request at the gateway layer, covering team, project, environment, model, and cost center — five dimensions that answer about 95% of allocation questions. If you can't tag reliably, you can't bill fairly, and you want to discover your tagging gaps while the stakes are a dashboard, not an invoice. Second, showback alone changes behavior: engineers who see their feature costs $4 per session start asking why before finance ever does. Practitioners consistently report that a month or two of showback surfaces the worst waste with zero enforcement.

Rung two: budgets. Give each team a token or dollar allocation with soft and hard limits. The standard implementation is two-tier: at 80% of budget, notify the team owner — nothing is blocked, the point is to surface the trend. At 100%, the gateway returns a 429 and refuses to call the provider, or degrades to a cheaper model. Budgets convert the capacity auction into an explicit negotiation: teams that want more must argue for it with usage data, not volume of complaint. Crucially, budgets cap downside without pricing individual experiments — inside your allocation, trying things is still free.

Rung three: chargeback. Actual money moves between cost centers. This requires finance integration, internal billing agreements, and organizational appetite for the fights that follow. Most companies need it eventually — it's the only mechanism that makes AI spend show up in the P&L where product decisions happen — but it only works when rungs one and two have made the numbers trustworthy. Chargeback against attribution data that teams dispute is a political tar pit.

The failure mode is skipping rungs. Chargeback-first means billing teams before they can see, predict, or control what they're being billed for. The predictable result is that every team's first response to their invoice is to attack the meter, not their usage.

Tokens Are the Wrong Unit to Bill

Even when you reach the chargeback rung, there's a design decision that matters more than the rollout sequence: what unit do you bill in? The default answer — tokens, at provider list price plus overhead — is the wrong one, for three reasons.

First, teams don't control tokens. A product team controls which features they ship and roughly how many tasks those features perform. They do not control how many tokens the platform's RAG pipeline stuffs into each context, how the shared agent framework structures its retries, or whether the platform routes them to a model whose responses are twice as verbose. Billing teams for a quantity the platform largely determines violates the first rule of chargeback: charge people for what they can change. When the platform team improves context caching and every team's bill drops 40%, that's the platform's win; when a framework upgrade doubles token burn, that's the platform's regression. Token bills scramble those signals.

Second, token prices are falling while token consumption explodes, which makes token-denominated budgets meaningless across time. A team whose bill is flat quarter over quarter might have doubled its delivered value on cheaper models, or halved it on pricier ones. The unit tells you nothing about the thing the business cares about.

Third — and this is the practical killer — token bills are unintelligible to the people who must act on them. A product manager staring at "2.3 billion input tokens, $18,400" has no idea whether that's good. The same PM staring at "41,000 resolved support conversations at $0.45 each" can immediately compare it to the $8 cost of a human-handled ticket and make a real decision.

The better unit is one level up the stack: outcome-adjacent work units. Charge per completed task, per resolved conversation, per document processed, per evaluated feature — whatever discrete thing the consuming team actually ships. The platform sets a rate per unit, informed by measured token costs but deliberately smoothed: the rate absorbs retry storms and model-verbosity variance, and it gets revised quarterly as the platform optimizes. This is exactly how the industry's own pricing is drifting — outcome-based pricing among AI vendors has jumped from a rounding error to a fast-growing share of contracts in under two years — and your internal economy should follow the same logic for the same reason: it aligns the bill with the value.

There's a bonus effect. Once you bill per task, the platform team becomes the natural owner of token efficiency, because the gap between the internal rate and the actual token cost is their margin to invest. Prompt caching, context pruning, and model right-sizing become platform-team wins that show up as widening margin, instead of optimizations nobody is incentivized to do because the costs pass straight through.

The Tell That Your Pricing Is Wrong: Shadow Integrations

How do you know when your internal pricing has drifted into dysfunction? There's one signal that dominates all the dashboards: teams start going around you.

A team that puts a corporate card on a public LLM API and builds a direct integration is telling you something precise — the internal platform's price, in money or in friction, exceeds the public market's. Sometimes it's literal price: your chargeback rate, padded with platform overhead, is higher than list. More often it's friction masquerading as price: a two-week approval to raise a rate limit, a gateway that adds 800 milliseconds, budget processes that can't accommodate a hackathon spike. Either way, your internal market has failed, and your most resourceful engineers — exactly the ones you want on the platform — are the first to defect.

This is the same dynamic security teams call shadow AI, and it is rampant: surveys find that virtually every large organization reports some unsanctioned AI use. The security framing treats it as a compliance problem to be blocked. The platform framing is more useful: shadow integrations are price signals. Every shadow deployment is a data point that your meter is charging more than the alternative, once you account for friction. Blocking egress without fixing the pricing just drives the behavior deeper underground — personal API keys, browser tabs, data pasted into consumer chatbots — where it carries real data-governance risk.

The countermeasure isn't enforcement first; it's making the sanctioned path cheaper than defection. The platform bundles things a rogue integration can't cheaply replicate — negotiated provider rates, compliance review, prompt-injection defenses, observability, failover across providers. If a team can rebuild all of that with a credit card and still come out ahead, your platform tax is too high. Audit the friction, not just the rate card.

A Working Design

Pulling the threads together, here's the shape that holds up in practice:

• Meter everything from day one, bill nothing yet. Gateway-level attribution with team, project, environment, model, and cost-center tags on every request. Public dashboards. You cannot retrofit attribution onto a quarter of untagged traffic.
• Run showback for one to two quarters. Let teams see their burn, fix the embarrassing waste, and learn to predict their own spend. Validate that the numbers survive scrutiny.
• Introduce budgets with soft/hard tiers. 80% alerts, 100% enforcement with graceful degradation to cheaper models before hard 429s. Give every team a real allocation — including an explicit, generous experimentation pool that is separate from production budgets, so prototypes are never competing with shipped features for tokens.
• When you graduate to chargeback, bill in work units, not tokens. Per task, per conversation, per document — priced off measured costs, revised quarterly, with the platform team keeping efficiency gains as margin.
• Watch for defection. Track shadow integrations as a pricing KPI, not just a security incident. Each one is telling you where the internal market failed.

The goal of all this machinery is not cost recovery — if the platform merely breaks even while strangling experimentation, it has failed. The goal is a functioning internal economy: one where the nightly batch job that quietly burns $40,000 a month has an owner who sees the number, where the promising agent prototype has free room to be wasteful while it's learning, and where "should we ship this feature?" can be answered with a unit cost and a unit value on the same line. Free inference can't produce that economy, and neither can a naive token meter. The ladder — showback, budgets, then outcome-priced chargeback — is how you get there without burning down the experimentation culture the platform exists to serve.