The Noisy Neighbor Is You: When Runaway Agents 429 Everyone Else on the Shared Account

The incident starts the way most of these do: a customer-facing feature throwing 429s in production at 2 p.m. on a Tuesday, no deploy, no traffic spike, nothing in the feature's own logs to explain it. The on-call engineer stares at the dashboard for twenty minutes before someone in another channel mentions, almost in passing, that they kicked off a "quick backfill" to re-summarize a few hundred thousand old documents. Two teams, two codebases, two on-call rotations — and one API key's worth of rate limit between them. The backfill ate the budget. The chatbot starved.

This is the noisy neighbor problem, and the twist that makes it so dangerous with model APIs is that the neighbor isn't some anonymous tenant on shared cloud hardware. The neighbor is another team at your own company, and the wall between you is thinner than anyone thinks.

The reason is structural. Provider rate limits — tokens per minute, requests per minute — are almost always scoped to the organization, not to the individual application, team, or job that happens to be making the calls. OpenAI's own guidance is explicit that adding more API keys under the same org does not increase your effective quota; the limit is a property of the account. That single fact reshapes everything downstream. Every service you run against that account is drinking from the same tap, and the tap has a fixed flow rate. When one service opens its mouth wide, the others get whatever's left, which on a bad day is nothing.

Per-Org Limits Make Every Agent a Shared-Fate Resource

In classic multi-tenant infrastructure, the noisy neighbor is a stranger. You and they landed on the same physical host by luck of the scheduler, one of you ran a runaway query or a CPU-bound loop, and the hypervisor let it bleed across the boundary. The industry spent a decade building isolation primitives — cgroups, ResourceQuotas, per-tenant statement timeouts — precisely so that one tenant's greed can't degrade another's latency.

Model APIs quietly reintroduce the problem one layer up, and they do it inside your own org boundary. There is no scheduler putting you next to a stranger. You put yourself there, deliberately, by pointing your chatbot and your nightly batch job at the same account. The token-per-minute budget is a shared, fixed pool, and consumption is fungible: a token spent summarizing a 2019 support ticket is a token that isn't available to answer a paying customer's question right now. The provider has no idea which request mattered more. It sees traffic against one org, and when the org exceeds its ceiling, it starts refusing — indiscriminately.

That fungibility is the whole trap. Compute isolation gives you a floor: my container is guaranteed its slice even if yours is on fire. A shared TPM budget gives you no floor at all. It's a single number, and whoever gets there first spends it. Interactive traffic — the stuff a human is waiting on — has no inherent priority over a background job that could just as easily run at 3 a.m. The API treats a user's blocked cursor and a backfill's thousandth retry as equal claims on the same scarce resource.

The Blast Radius of a Single Concurrency Setting

The scary part of this failure mode is how small the triggering change is. Nobody sets out to DoS their own production. What happens instead is that someone bumps a concurrency knob from 4 to 40 in a batch worker, because the batch was running slow and 40 seemed fine on their laptop. That one integer is the blast radius.

Do the arithmetic that nobody does beforehand. Say your org's ceiling is a few hundred thousand tokens per minute — comfortable for the interactive product, which sips a few thousand tokens per request at human pace. Now a batch job spins up 40 parallel workers, each firing multi-thousand-token prompts as fast as the socket allows. Forty workers with no pacing can saturate a six-figure TPM budget in seconds, not minutes. The batch doesn't need to be malicious or even large. It just needs to be unpaced, and the default posture of most job code is exactly that: loop over the work list, call the API, go as fast as you can.

From the interactive product's side, this is indistinguishable from an outage. Its own error rate spikes, its own latency climbs as its own retries queue up, and its logs show 429s with no local cause. The team that owns it burns an hour ruling out their own deploys, their own dependencies, their own region, before anyone thinks to ask what else is running on the account. The incident's root cause lives in a repository the responders don't have checked out. That cross-team invisibility is why these incidents run long: the signal and the cause are owned by different people who aren't in the same room.

Why "Just Add Backoff" Doesn't Save You

At this point someone always says the reassuring thing: we have retries with exponential backoff, the client will smooth this out. It won't, and understanding why is the difference between a system that recovers and one that oscillates.

Loading…