Reserved Capacity for Tokens: The Reserved-Instance Decision Nobody Ported From the Cloud Era

Most teams buy inference the way their predecessors bought EC2 in 2010: everything on-demand, metered to the token, and surprising on two axes at once. The bill is one surprise. The rate limit is the other — the 429 that shows up in the middle of a launch, right when the on-demand pool you never reserved gets contended by everyone else who also chose to pay retail. Then someone opens the pricing page and discovers that the provider has been quietly selling reserved capacity the whole time: provisioned throughput, committed-use discounts, dollars-per-unit-per-hour instead of dollars-per-million-tokens. The reserved-instance decision that took the cloud industry a decade to internalize is sitting right there for tokens, and almost nobody has ported the playbook.

The reason isn't ignorance. It's that the reserved-instance math you learned for compute doesn't transfer cleanly, and the ways it breaks are exactly the ways that punish a naive commitment. A reserved EC2 instance is a bet that you'll still want that instance type in a year. A reserved block of token throughput is a bet that you'll still want that model in a year — and models have a shelf life measured in months, not decades. The commitment structure is familiar. The thing you're committing to is not.

This post is about the decision, not the vendor. The mechanics differ across Azure's provisioned throughput units, AWS Bedrock's provisioned throughput, and the committed-use tiers other providers offer, but the shape is the same everywhere: you trade flexibility for a lower unit price and a guaranteed capacity floor. The interesting questions are which parts of the classic reserved-versus-on-demand analysis survive the move to tokens, which parts invert, and what FinOps muscle you need to build before a commitment is a smart move instead of an expensive one.

What You're Actually Buying

Start with the billing model, because it's the thing people misread. On-demand inference is metered on consumption: you pay per input and output token, and a completely idle hour costs nothing. Reserved capacity inverts that. You buy a fixed number of throughput units — Azure calls them PTUs, Bedrock calls them model units — and you're charged for them by the hour whether or not a single request flows through. Azure's own documentation is blunt about it: "you're billed for the full deployed PTU count regardless of actual utilization," and a provisioned deployment "can't be paused. Billing stops only when the deployment is deleted."

That single sentence reframes the entire decision. On-demand, your cost tracks your traffic. Reserved, your cost tracks your reservation, and your job is to keep that reservation busy. The unit economics only work if the capacity you bought is actually doing work. This is the reserved-instance bargain restated for inference: you get a lower effective price per token and a capacity guarantee, but you've converted a variable cost into a fixed one, and fixed costs are only cheap when utilization is high.

The discounts are real and worth the trouble at scale. Provisioned tiers commonly land 20–40% below on-demand at sustained load, and stacking a term reservation on top — Azure's one-month or one-year commitment, Bedrock's one-month or six-month terms — pushes the effective rate lower still, with annual commitments running roughly 30% under monthly pricing. The entry points are non-trivial, though. Azure's Global and Data Zone provisioned deployments start at a 15-PTU minimum; regional deployments want 25 to 50. Bedrock provisioned throughput is billed in the tens-to-low-hundreds of dollars per hour per model unit. These are not knobs you turn on for a side project. They're commitments sized for a workload you already understand.

The Break-Even Point Is Higher Than Your Intuition

Everyone wants a single number, so here is the honest version: the break-even sits somewhere around 60–80% sustained utilization of the capacity you reserve. Below that, on-demand is cheaper even at a higher per-token rate, because you're not paying for idle units. Above it, the reservation wins and keeps winning as you push utilization toward the ceiling.

Translating that to volume depends on the model, but the thresholds are large. Public break-even estimates for a mainstream model like GPT-4o land in the neighborhood of 150–200 million tokens per month before a PTU deployment beats pay-as-you-go. That's the number that stops most teams cold: if your workload isn't already pushing hundreds of millions of tokens a month with a flat profile, the commitment is a loss. And "flat" is doing a lot of work in that sentence.

The utilization math is unforgiving in a specific way. Reserved capacity is priced against a 24/7 hour count, but almost no product generates traffic 24/7. A B2B tool with a weekday-business-hours load pattern is idle roughly two-thirds of every week — nights, weekends, holidays. If you size a reservation to your peak hour, your average utilization against the always-on meter might be 25%, and you've just paid retail-plus for the privilege of a capacity floor. This is why the good candidates for reservation are the workloads with genuinely flat demand: steady weekday chat products at global scale, voice agents with predictable concurrent-session counts, batch pipelines you can schedule to fill the trough. The bad candidates are spiky internal tools, experimentation traffic, and anything whose load chart looks like a mountain range.

The corollary is a posture, not a purchase: reserve the floor, burst on-demand. Size your reservation to the baseline you're confident you'll always consume — the flat bottom of your traffic chart — and let the peaks spill over to on-demand or to a cheaper fallback model. Azure and Bedrock both bill any usage above your reserved units at the standard hourly or per-token rate, so a hybrid deployment isn't a hack; it's the intended design. You capture the discount on the predictable base and keep the flexibility where demand is genuinely variable.

Loading…