Your AI Bill Is One Untagged Line Item: FinOps When Tokens Refuse to Be Tagged

Finance opens the invoice at the end of the month. One vendor. One number. It is bigger than last month, and it will be bigger next month. Then they ask the only question that matters — which feature spent this? — and nobody in the room can answer.

This is the quiet failure mode of running AI in production. Not that the bill is large; large is fine if the value is there. The failure is that the bill is unattributable. It arrives as a single line item — OpenAI, Anthropic, Bedrock, Azure — with none of the dimensions finance actually needs: not per feature, not per team, not per customer, not per successful task. You can see the total go up. You cannot see why, and by the time the invoice lands, the request context that could have explained it is long gone.

The reason this catches good engineering teams off guard is that they already solved cost attribution once, for cloud. Every EC2 instance, every S3 bucket, every managed database is a resource with an ARN you can tag. team:payments, env:prod, service:checkout — attach the tags, and the cost-and-usage report slices itself. A decade of FinOps tooling assumes this model: costs hang off taggable assets, and allocation is a reporting problem downstream of tags that already exist.

An LLM API call breaks that assumption at the root. It is not an asset. It is a transaction — a stateless request that returns tokens and vanishes. There is no resource ID to tag, no long-lived thing to hang a label on. The provider bills you by API key and by time, aggregated into a total. Whatever business context existed at the moment of the call — which user, which feature, which customer, which agent run — is not in the billing data unless you put it there, at the moment of the call, before the request leaves your process.

Why token spend resists the tags you already have

The structural problem is that the context lives in the wrong place. Your LLM calls happen deep inside functions that have no idea what business action they are serving. A summarize() helper does not know whether it was invoked by the document-search feature or the email assistant. A retrieval step does not know which customer's request it is fulfilling three call frames up. The token cost is generated at the bottom of the stack; the meaning lives at the top.

Cloud tagging works because the meaning and the resource are the same object — you tag the database, and every query against it inherits the attribution for free. LLM calls invert this. The thing that costs money (the request) and the thing that explains the cost (the business intent) are separated by the entire call stack, and nothing carries the intent down unless you explicitly thread it through.

This is why the two obvious escape hatches disappoint. Provider-native attribution — project-scoped keys, workspace usage APIs, per-key cost breakdowns — is real and worth turning on, but its granularity stops at the key. You get "this key spent $40k," not "the onboarding flow spent $40k, mostly on one enterprise tenant's oversized documents." And you cannot mint a key per feature per tenant per environment without drowning in key management. Retroactive log parsing is worse. Once the request is done, your logs show an opaque model call: provider, model, token count. The feature that routed it, the user who triggered it, the agent run it belonged to — that context was in memory at call time and is simply not recoverable from the log line afterward. Reconstruction is slow, lossy, and always under-reports the exact cases you care about most, like long agentic traces that sprawl across dozens of calls.

Attribution is only honest when the tags are attached at request creation time. There is no later.

What you actually have to instrument

The fix is unglamorous: attach business metadata to every model call, at the call site, and propagate it through every hop. Practitioners have converged on a small set of fields worth carrying on each request:

• feature — the product surface (chat, summarization, agent_research). This is the dimension finance and product both want first.
• customer_id (hashed) — enables per-tenant unit economics and margin protection in B2B.
• user_id — isolates the heavy accounts distorting your averages.
• agent_run_id — groups a multi-step execution so a runaway loop shows up as one expensive trace, not fifty cheap-looking calls.
• prompt_version — pins the template so a cost regression can be traced to the edit that caused it.
• deployment — keeps staging spend out of your production unit economics.

Two implementation paths carry these without turning every function signature into a metadata bucket brigade. The first is an LLM gateway or proxy — LiteLLM, Portkey, Helicone and similar — that every model request routes through. Because it is a single chokepoint, it prices, tags, logs, and budget-checks every call in one place, and you inject the metadata as request headers rather than plumbing it through application code. The second is trace-context propagation via OpenTelemetry: a trace started at the HTTP boundary already carries user_id and session_id, and any LLM span created inside that trace inherits them automatically, without each intermediate function passing them down by hand. Most mature setups use both — the gateway for enforcement and authoritative pricing, the trace for the deep call-graph context a gateway header can't see.

Loading…