Capacity Planning for Agents: Why Concurrency, Not QPS, Is Your Real Unit
The first time an agent product falls over in production, the on-call engineer usually pulls up the request-rate graph and finds it boringly flat. Forty requests per second, same as yesterday. No spike, no surge, no obvious cause. Meanwhile the system is on fire: timeouts everywhere, the provider returning 429s, the queue depth climbing in a straight line toward the moon. The graph that should explain the outage looks completely healthy, because the graph is measuring the wrong thing.
QPS — queries per second — is the unit we inherited from a decade of stateless web services. A request comes in, you do tens of milliseconds of work, you send a response, the request is gone. Under that model, request rate and load are nearly the same number, and capacity planning is mostly "how many requests per second can one box handle, multiply by boxes." That mental model is so ingrained that we reach for it automatically. For agents, it quietly lies to you.
