The Tool Belt That Grew a Long Tail Nobody Uses

Nobody decides to give an agent forty tools. It happens the way a garage fills up. You wire in a search tool, then a database reader, then someone on the team ships a Slack integration, then the ticketing MCP server gets installed because it was one config line away. Each addition is individually reasonable. Nobody ever removes anything, because removing a tool feels like taking away a capability, and taking away a capability feels like a regression.

Six months later your agent has a tool belt with three tools it uses constantly, a dozen it uses occasionally, and a long tail of two dozen it has technically never selected in production. That long tail is not free. It is not even cheap. Every unused tool in the catalog is actively making the agent worse at choosing among the ones that matter.

This is the counterintuitive part that trips up otherwise careful teams. We reason about tools the way we reason about library functions: an unused import costs nothing at runtime, so more available options can only help. But a tool an LLM never calls is nothing like an unused import. It sits in the context window on every single request. It competes for the model's attention on every single decision. And it widens the space of wrong answers the model can wander into. The cost is not paid when the tool is used. It is paid on every request when the tool is not used.

The Numbers Are Worse Than Your Intuition

The failure mode has a clean shape once you measure it, and the measurements are not subtle. Across benchmarks and production reports, tool-selection accuracy degrades as the catalog grows, with documented performance losses ranging from single digits to catastrophic collapses depending on how similar the tools are and how the descriptions are written. Evidence from multiple tooling vendors converges on the same inflection: performance starts degrading sharply somewhere past twenty tools, and the major agent runtimes have hard ceilings — some cap you at forty tools total, others at 128 — precisely because they know what happens beyond that.

The two mechanisms behind the decline are worth naming because they call for different fixes.

The first is context bloat. Every tool description is tokens, and detailed descriptions are a lot of tokens. Reports from teams running large MCP setups describe tool definitions consuming 50,000-plus tokens before the agent has read a single word of the user's actual request. One documented case had 58 tools eating roughly 55,000 tokens; setups north of 134,000 tokens of pure tool schema have been observed in the wild. That is context the model no longer has for reasoning, for conversation history, or for the retrieved documents that were supposed to be the point. You paid for a big context window and spent it describing tools the agent will not call.

The second is attention dilution, and it is the more insidious one because it does not show up on your token bill. Even with unlimited context, presenting many tools — especially several that look alike — degrades the model's ability to pick correctly. The model spreads its attention thin, hesitates between near-duplicates, hallucinates parameters, and occasionally enters what practitioners call a doom-loop of indecision, calling tool after tool without converging. Two tools named search_documents and search_knowledge_base are a trap. The model cannot reliably tell them apart, and neither, frankly, can the engineer who added the second one.

Why "Just Add It, It Might Be Useful" Is a Statistical Mistake

Here is the framing that changes how you think about the catalog. The right question is not "does this tool help when it is the correct choice?" It is "how much does having this tool available help the model beat random guessing?"

Recent work on chance-corrected tool selection makes this concrete with a metric worth internalizing: measure selectivity in bits over random. Showing five tools out of a five-hundred-item registry gives the model roughly 3.3 bits of selectivity to work with — a genuinely favorable ratio. But when a task has three or four relevant tools buried in a pool of fifty-eight, even a perfect selector operating on that raw pool earns only about 0.02 bits over random chance. The signal is drowned. The correct tool is present, but the model's odds of reaching for it are barely distinguishable from a coin flip across the whole belt.

Flip that around and the design principle falls out. Selection accuracy is not a property of the tool. It is a property of the ratio of relevant tools to distractors that the model sees at decision time. A validation with a current-generation model showed the model picking the correct tool 93 percent of the time when the tool set was adaptively narrowed, versus 87 percent when it was handed a fixed larger list — a six-point swing bought purely by removing distractors. The tool it needed was present in both cases. The only difference was how much noise it had to see it through.

So when someone says "let's add this tool, it can't hurt," they are wrong in a specific, quantifiable way. Adding a tool that fires on two percent of requests improves those two percent and taxes the other ninety-eight. Unless that tool is doing something genuinely irreplaceable, the trade rarely clears.

Loading…