2 posts tagged with "unit-economics"

A team I know cut their inference bill 40% last quarter by migrating their support-email triage flow from a frontier model to a mid-tier one. The CFO sent a thank-you note. Six months later, customer support headcount was up two FTEs and average resolution time had risen 35%. Nobody connected the dots, because the dots lived in different dashboards: the inference bill on the platform team's, the support load on the operations team's. The migration looked like a win on the only metric anyone was tracking. The metric was wrong.

This is the cost-per-token trap. Your invoice tells you what you spent on tokens. It cannot tell you what you spent per correct task, because the inference vendor has no idea what "correct" means in your domain. They sold you raw compute. You bought outcomes — or thought you did. The gap between those two units is where AI unit economics quietly comes apart, and the team that doesn't measure the right denominator is running half the equation and shipping the other half blind.

A single developer ran up more than $35,000 in compute under a $200 monthly plan. That is a 175x subsidy on one user — paid for by the casual majority who would have been just as happy on a $19 tier. This is the load-bearing math behind every "Why is our AI margin negative this quarter?" Slack thread. The problem is not that one user; it is that the long tail of one users follows a power law, and a power law plus flat-rate billing plus a real per-unit cost is a structural margin compressor that no amount of growth will fix.

The reflex when this lands on a finance review is to clamp down: hard token caps, "fair-use" language buried in the TOS, weekly throttles, a quietly degraded model for free tier. These all work in the sense that they cut the bleed. They also alienate the exact users whose evangelism you depend on, because the people who hit your caps are the ones who actually figured out how to extract value from your product. The standard fix is a backwards-compatible apology to the wrong cohort.