Your Agent Is a Chatty Client: Data Gravity Comes for the Tool Loop

Fifteen years ago, we learned to fear the N+1 query: an ORM that looked innocent in code review would fire one query for a list and then one more query per row, and a page that should have made two database calls made two hundred. We fixed it with eager loading, batching, and a generation of linters. Then we built AI agents, and we shipped the same bug at a much more expensive layer.

A single agent task — "reconcile these invoices," "triage this incident" — routinely makes dozens of serial tool calls. Each one is a full network round trip: agent to tool, tool to data store, data back through the tool, result serialized into the model's context, another inference pass to decide the next call. If your inference runs in one cloud and your data lives in another, every one of those hops crosses a metered, high-latency boundary. The dominant cost term of your agent system is no longer the model. It's the geography.

Nobody itemized this. Latency budgets were written per call, egress was a rounding error on the storage bill, and the model invoice got all the scrutiny. Meanwhile the tool loop quietly became the chattiest client your infrastructure has ever served.

The N+1 Problem, Reborn at the Tool Layer

The structure of the failure is identical to the ORM version, just relocated. The agent doesn't know your data model. It knows tools. So instead of asking one well-shaped question, it asks a sequence of small ones: fetch the ticket, then fetch the customer, then fetch the customer's plan, then fetch the plan's limits, then check the usage table. Five round trips to answer a question a SQL join would have answered in one.

The ORM version of this bug cost you milliseconds inside a datacenter. The agent version costs you three things per hop:

• Network time, which is now cross-cloud instead of cross-rack. A modest agentic loop makes five to ten round trips per task; at 50 milliseconds of cross-region latency per hop, that's 250 to 500 milliseconds of pure network tax before any inference or tool execution happens.
• An inference pass, because after every tool result the model must read the full accumulated context and decide what to do next. Three round trips means roughly three times the input tokens processed — re-sent context has been measured at around 62 percent of total agent inference bills.
• Egress, because every byte the tool returns across a cloud boundary is metered at $0.09 to $0.12 per gigabyte on the major providers — often more than the storage itself costs.

Each of these individually looks tolerable. The bug is that they multiply per call, and agents make a lot of calls.

Why Per-Call Latency Budgets Mislead

The standard operational guidance for tool servers is sensible on its face: keep p50 under 50 milliseconds, p95 under 200. Teams hit those targets, dashboards go green, and the agent still takes ninety seconds to complete a task. The budget was measured at the wrong altitude.

Per-call percentiles describe a single hop. An agent task is a tree of hops — often forty deep once you count retrieval, sub-agent delegation, and verification steps — and the hops are mostly serial, because each tool result feeds the model's decision about the next call. Wall-clock time is the sum along the longest path, not the p95 of any node on it. A tool fleet where every server meets a 200-millisecond p95 can still produce a task that spends eight seconds purely in transit, because 40 sequential hops at even 200 milliseconds is eight seconds of network before a single token is generated.

Tail behavior makes this worse than the arithmetic suggests. With forty serial calls, the probability that at least one lands in the p95 tail is near certainty — the task-level latency distribution is dominated by per-call outliers you declared acceptable. And when a slow call turns into a timeout, the agent retries, which re-sends the full conversation context: a three-retry pattern on one database read triples the token cost for that step. The retry loop converts a latency problem into a billing problem.

The metric that actually predicts user experience and cost is task-level: round trips per task, network milliseconds per task, tokens re-processed per task. If you only track per-call percentiles, your agent can get slower and more expensive every sprint while every dashboard stays green.

The Egress Line Item That Makes Co-Location Math Explicit

Data gravity used to be an argument about analytics: your petabytes are in one cloud, so your data warehouse ends up there too. Agents sharpen the argument because they don't read data once — they live in it. Context is re-fetched across every reasoning cycle: conversation history, policy documents, retrieval results, tool schemas. The data isn't an input to the workload; it largely is the workload.

That changes the economics of the split-cloud architecture that many teams backed into. The pattern is common because it was locally rational at every step: the GPUs (or the model API) are in cloud A because that's where capacity or pricing was best; the operational data is in cloud B because it's been there for a decade. Each agent task now pumps tool results across that boundary dozens of times, and the meter runs in both directions of your attention — latency on every hop, egress on every byte.

The egress fee is, in a strange way, the honest part of the system. Latency tax is invisible until you instrument it; token re-processing hides inside a bill you already expected to be large. But egress shows up as its own line item, priced per gigabyte, attributable to a specific boundary crossing. It's the one number that forces the co-location question into the open: 55 percent of IT leaders already cite egress costs as the biggest barrier to switching providers, and specialized GPU clouds have started marketing zero-egress tiers precisely because the line item has become a deal-breaker. When inference crossed half of total AI cloud infrastructure spend in early 2026, placement stopped being a procurement detail and became the architecture.

Placement Is a Design Decision with Three Honest Options

Once you accept that the tool loop is a chatty client, the response space is small. You can move the tools to the model, move the model to the data, or restructure the loop so there are fewer round trips to place. Most real systems need a mix.

Move the tools to the model. Replicate or cache the hot slice of data next to inference: a read replica in the model's region, a vector index co-located with the GPU pool, a policy-document cache that refreshes hourly. This works when the agent's working set is small and read-heavy, which is more common than teams assume — most agent tasks touch a narrow band of reference data over and over. The cost is a consistency story you now have to own.

Move the model to the data. Run inference in the cloud and region where the data already sits, even if the GPU pricing is worse or the model selection narrower. This is the data-gravity answer, and for regulated data it's often the only answer — sovereignty and compliance constraints fix the data's location before any architectural discussion starts. Paying 20 percent more per GPU-hour to eliminate forty cross-cloud hops per task is frequently the cheaper trade once you price the hops honestly.

Batch the round trips away. The most underused option: restructure the loop so intermediate results never transit the model at all. Instead of the model orchestrating twenty serial tool calls — each one a network hop plus an inference pass — the model writes a small program that executes next to the tools, and only the final answer returns to context. Anthropic's code-execution-with-MCP pattern demonstrated a representative workflow dropping from 150,000 tokens to 2,000, eliminating nineteen inference passes along the way. Where calls are genuinely independent, parallel dispatch collapses their latency from the sum to the max — orchestration-layer research like LLMCompiler reports latency gains up to 3.7x and cost reductions up to 6x over serial loops. This is eager loading for agents: same cure as the ORM era, one layer up.

The wrong answer is the default one — leaving placement implicit, letting each team pick its cloud independently, and discovering the topology of your agent system for the first time on an invoice.

Budget the Task, Not the Call

The practical shift is to treat an agent task the way you'd treat a page load in the N+1 era: as the unit that gets a budget.

Give each task class a wall-time budget and a round-trip budget, and make the tree visible — trace tool calls end-to-end so you can see the longest serial path, not just per-server histograms. Track egress per task as a first-class cost metric next to tokens per task; the ratio between them tells you whether your problem is chattiness or placement. When a task blows its budget, the fix is usually structural, not incremental: collapse five narrow tools into one that answers the actual question, push a loop into co-located code execution, or move the data.

The teams that internalized N+1 didn't get there by making each query faster. They got there by making the query count a reviewable property of the code. Agents need the same discipline one level up: round trips per task is a number someone should see in review, with a comment required when it grows. Data gravity is not going to weaken — models get faster, data gets heavier, and the boundary between them gets more expensive to cross. The agent systems that stay economical will be the ones that were designed, from the start, around where the data actually lives.