The Off-Hours Cost Curve: Why Your AI Feature Spends Differently on Saturday Than on Tuesday

The cost dashboard everyone looks at is a weekly rolling average, and that average is lying to you. Not in the sense that the number is wrong — it's a faithful arithmetic mean of a billing event stream — but in the sense that it is hiding the shape of the cost curve underneath. The hours between Friday evening and Monday morning consume tokens differently from the hours between Tuesday at 10am and Thursday at 4pm. The cohort active on Saturday at 3am is not the cohort active on Tuesday at 11am, and the per-user economics of those cohorts diverge by a factor that nobody writes down because the dashboard averaged it away.

Most teams discover this the first time a weekend automation script melts the budget. A LangChain agent gets into an infinite conversation cycle Friday night, runs for the better part of a week before anyone notices, and produces a five-figure invoice that has to be explained to finance on Monday morning. The post-incident review treats it as a one-off — bad retry logic, missing budget cap, didn't page on-call. But the same dashboard that hid the runaway loop is also hiding the steady-state version of the same phenomenon: a baseline of off-hours traffic whose unit economics are structurally worse than the business-hours baseline, every single week, and which the weekly average smooths into invisibility.

The Mix Is the Curve

When a product team thinks about "weekend traffic," they typically picture the same users using the same product at a different time. That mental model is wrong for AI features. The user mix at 3am Saturday is not a time-shifted version of the user mix at 11am Tuesday — it is a different population. The composition typically includes some combination of insomniacs willing to spend an hour wrestling with a chatbot they wouldn't tolerate during a focused workday, international users in time zones the product wasn't designed for, prosumer power users running their own scripts against the product API, jailbreakers and red-teamers probing for refusal patterns, abuse bots driving traffic that looks human enough to slip past basic filters, and automation runs scheduled to fire during cheap-compute windows because the operator believes the provider has off-peak pricing.

Each of those sub-populations has a different token economy. The insomniac sends long, exploratory queries with low conversion intent. The prosumer hits the same endpoint a thousand times an hour and expects every call to succeed. The jailbreaker iterates on adversarial prompts that are designed to drive the model into longer, more expensive completions. The abuse bot pastes a 50KB document and asks for a paragraph back. None of these resemble the median Tuesday-morning user whose query is short, whose intent is concrete, and whose session ends after three turns. When the cost-per-active-user metric is computed as a daily or weekly aggregate, these populations are blended into a single average that flatters the business-hours cohort and absorbs the off-hours cohort into the noise.

The blended number is not just imprecise — it is structurally misleading in a way that affects pricing. A flat-rate plan priced against the blended average is overcharging the business-hours user (who is implicitly subsidizing the off-hours cohort) and undercharging the off-hours cohort (whose marginal cost exceeds their marginal revenue). The pricing team sees stable gross margin and concludes the unit economics are healthy. The cost team sees stable daily spend and concludes nothing is on fire. Both are correct about the aggregate and wrong about every cohort the aggregate is composed of.

What the Rolling Average Hides

A 7-day or 30-day rolling cost average has three structural blind spots, all of which compound for AI features specifically.

The first is the temporal mix shift. Cost per user-hour rises sharply in the early-morning hours of any weekend because the legitimate-user denominator collapses faster than the automation-and-abuse numerator. A weekend graveyard shift of 50 users including 10 abuse bots and 5 runaway scripts produces a wildly different cost-per-user than a weekday peak hour of 50,000 users with the same five runaway scripts. Plotting cost-per-user against hour-of-week produces a heatmap with bright spots at the boundaries of business hours and in the deep weekend — bright spots the weekly average converts into a single dim number.

The second is the cost-slope inversion. During business hours, cost rises roughly in proportion to active users. After hours, cost rises roughly in proportion to elapsed time, because automation and abuse traffic don't sleep. A team that alerts on "today's spend versus yesterday's spend" will miss a runaway weekend automation because each day's total looks normal — the spike is distributed across 48 hours rather than concentrated in one. Alerting on the slope of cost-per-elapsed-hour, segmented by time-of-week, is what catches the LangChain-loop pattern before Monday standup.

The third is the cache-hit rate collapse. AI features increasingly rely on prompt caching to make the unit economics work. Cache locality depends on a small number of high-traffic prompt prefixes being repeated thousands of times per minute. In off-hours, request volume drops, the cache evicts hot entries, and the next morning's wake-up traffic pays full input-token price for prefixes that would have been cached at business-hours volume. The cost dashboard reports an average input-token cost across the day; nobody sees that the 6am-to-9am window paid 4x the median rate per token because the cache was cold.

Why the Org Doesn't Catch It

The off-hours cost curve persists in part because no single team owns the seam between the engineering metrics and the finance metrics. Engineering looks at request volume, latency, and error rate. Finance looks at monthly invoice and gross margin. Neither is asked to look at cost-per-cohort-hour, because the question lives between their two charters.

Loading…