209 posts tagged with "agents"

Your agent fleet opened forty pull requests this week. You reviewed the six that touched payment code, skimmed a few that happened to land while you had the tab open, and merged the rest on green CI. If someone asked you what your review policy is, you would describe something like this — and it is not a policy. It is a mood.

The numbers say most teams are in the same place. A recent large-scale study of agent-authored pull requests in popular open-source repositories found that 61% receive no recorded review activity at all, and of the ones that do get reviewed, most are reviewed only by other agents. Meanwhile the volume keeps climbing: agents now produce PRs, documents, support responses, and tickets at a rate no human review process was designed for. Reviewing everything is impossible. Reviewing nothing is malpractice. So teams improvise somewhere in the middle, with no stated rule, no measured coverage, and no way to say whether the current level of scrutiny is paranoid or reckless.

Manufacturing solved this exact problem a century ago. When Western Electric was churning out telephone equipment in the 1920s, inspecting every unit was economically impossible and shipping uninspected lots was unacceptable — so Bell Labs statisticians built acceptance sampling: a mathematically grounded discipline for deciding how much of a batch to inspect, when to reject the whole batch, and when a supplier has earned lighter scrutiny. It became MIL-STD-105 during World War II, then ANSI/ASQ Z1.4 and ISO 2859-1, and it still governs how a container of goods gets accepted at a port today. The mapping onto agent fleets is almost embarrassingly direct — and almost nobody in AI engineering has picked it up.

Every annual planning cycle in every engineering org runs on the same hidden equation: roadmap ambition divided by engineer output equals requisitions. It has been true for so long that nobody writes it down anymore. You size the work, you divide by what a team can ship in a year, and the remainder becomes a hiring plan. Finance builds the budget around it, recruiting builds pipelines around it, and managers build careers around it.

That equation quietly broke. In an agent-heavy org, the marginal unit of engineering output is no longer another senior hire — it is tokens plus the review bandwidth to absorb what those tokens produce. NVIDIA now hands engineers token budgets worth roughly half their base salary, and Jensen Huang has said he would be "deeply alarmed" if a $500,000 engineer consumed less than $250,000 of tokens a year. Whether or not you take the specific ratio seriously, the structural point stands: a company can now convert dollars into working code through two different doors, and the annual plan only has a form field for one of them.

Every team building agents eventually hits the same wall. The system prompt started at 400 tokens. Then someone added the database migration checklist. Then the postmortem template, the deploy runbook, the style guide for customer emails. Eighteen months later it's a 9,000-token monolith that nobody dares to edit, because changing the line about rollback procedures somehow degrades the agent's tone in support tickets. You've built the prompt equivalent of a single 50,000-line main.c — and everyone is statically linking against it.

The instinct is to reach for RAG: chunk the runbooks, embed them, retrieve on demand. That fails in a subtler way. RAG is built to retrieve facts, and facts degrade gracefully when fragmented — three out of five relevant chunks about your billing model still tells the agent most of what it needs. Procedures don't degrade gracefully. A database migration runbook retrieved at 60% is not 60% useful; it's a production incident. Steps 1 through 4 without step 5 ("verify replication lag before cutting over") is worse than no runbook at all, because the agent now acts with confidence it hasn't earned.

Pull up your token dashboard and ask one question about every workload on it: was a human waiting for this response? For most teams running agents in production, the honest answer is no for half the bill or more. Eval suites, embedding backfills, nightly report generation, bulk classification, overnight code migration, summarization of yesterday's tickets — none of it has a user staring at a spinner. Yet nearly all of it flows through the interactive endpoint, at full price, competing for the same capacity as the requests that actually are latency-critical.

Every major provider will run that deferrable work for half the cost. OpenAI's Batch API, Anthropic's Message Batches, and Gemini's batch mode all price asynchronous jobs at a flat 50% discount in exchange for a 24-hour completion window. The discount requires no negotiation, no committed spend, no engineering heroics. It requires only that you admit, in your architecture, that some work can wait — and most teams have never made that admission, because nobody made deferral a design decision.

We have seen this movie before. Spot instances offered 60–90% off cloud compute for years while most teams kept everything on on-demand, not because the savings were unreal but because using them forced an uncomfortable question: which of our workloads can tolerate interruption? Teams that answered it built checkpointing and cut their compute bills by more than half. Teams that didn't kept paying the "everything is urgent" tax. The batch tier is the same fork in the road, except the axis is latency tolerance instead of interruption tolerance — and agentic workloads, which burn 5–30x more tokens per task than chatbots, make the price of not choosing much steeper.

When a dependency breaks, your build fails. When a workaround becomes unnecessary, nothing happens at all. That asymmetry is why every production LLM system older than a year is carrying scaffolding it no longer needs — retry choreography, output-repair parsers, forced chain-of-thought, elaborate task decomposition, chunking heuristics — each one built as a compensation for a specific model's specific weakness, and each one silently outliving the weakness it compensated for.

The uncomfortable part is that this isn't a hygiene problem, like stale feature flags. Obsolete scaffolding doesn't just sit there costing you latency and tokens. In the worst case it actively constrains the new model to the old model's ceiling: your decomposition logic chops a task into six steps because the 2024 model couldn't hold the whole thing, and the 2026 model — which could have one-shotted it — now inherits six opportunities to lose context at the seams you built.

Count the readers of your stack traces. For most internal tools, the answer used to be "one tired engineer, occasionally." Today the highest-volume reader of your error output is almost certainly a language model inside a retry loop. Coding agents read your linter warnings, your CLI usage strings, your API error bodies, and your test failures thousands of times a day — far more often than any human ever will. And unlike the human, the agent takes every word literally.

That changes what an error message is. It is no longer documentation of a failure. It is an instruction injected into the context window of the next attempt — a prompt you wrote months ago, now steering fleets of agents you've never met. A precise error converges the loop in one retry. A vague or misleading one sends the agent spiraling: wrong fixes, --no-verify workarounds, hallucinated flags, burned tokens. If you maintain a tool, a service, or a build system, you are already doing prompt engineering. You're just doing it in your error strings, and probably by accident.

The quarter your team adopted coding agents, two things happened that nobody put on the same slide. Throughput went up — more pull requests, more merged code, more tickets closed. And your three most senior engineers got slower. Not lazy-slower. Drowning-slower. Their own commits dried up, their calendars filled with review, and the one-on-one refrain shifted from "here's what I shipped" to "I spent the week unblocking everyone else."

This is the seniority inversion. The people whose time was supposed to get more leveraged by AI are the ones who got buried by it. And it isn't a motivation problem or a tooling gap. It's a structural consequence of what agents actually do to the shape of work: they make generation cheap and verification expensive, and verification is the one task you can't hand to a junior or to another agent.

There is a class of agent bug that does not appear in any single component when you look at it in isolation. The model is fine. The tool is fine. The retry policy is fine. The timeout values are even, on paper, generous. And yet a tool that consistently completes in eight seconds keeps landing against an agent that has already declared it a failure at seven point nine, replanned around an "error" that never happened, and started a second call that the first call's result is about to collide with.

The bug is not in any of the boxes. It is in the gap between two clocks that nobody agreed should be the same clock.

The user hits stop. The browser closes the SSE connection. Your AI SDK fires onAbort. The agent runtime sees the signal, stops requesting more tokens from the model, and tears down its loop. From inside your codebase, the cancellation looks crisp. Every subsystem you can see is doing the right thing.

Meanwhile, two seconds earlier, the model emitted a tool call. The runtime dispatched it. The tool's execute function opened a TCP connection to a third-party API and posted a payload. That HTTP request is still in flight, the third party's server is still processing it, and the third party has no way of knowing that the conversation it is serving no longer exists. The write commits. The user's mental model says they escaped the action by hitting stop. The downstream system's database says otherwise.

A new engineer joined the team on Monday. By Wednesday, she had a working local agent setup: an MCP server bridged to the company's deployment API, pointed at staging, talking to her editor. The onboarding doc walked her through the OAuth flow. The token she pasted into the server's environment file was the one her teammate had emailed her — the same token the CI pipeline uses to ship to staging. By Friday, she had joined the team for a working session at a coworking space.

The MCP server was still running. Bound to 127.0.0.1. No authentication. The token was loaded into the process. She didn't think about it because she was not using it. But any tab that visited any website that day could speak to her local server through her own browser. So could any other laptop on the coworking wifi, because she had not noticed that the server was actually bound to 0.0.0.0. The OAuth token your CI pipeline uses to push to staging was now reachable by anyone who could trick a browser into making a request to a local IP — which, in 2026, is a one-pop-up problem.

This post is about that class of failure: the gap between "I'm developing on my laptop" and "my laptop is a server reachable by adversaries." MCP servers, by design, sit right in that gap. Most teams have not noticed.

A customer support team escalates: "Your assistant used to handle refund-eligibility questions correctly. Last week it started getting them wrong." The on-call engineer pulls a transcript, replays the exact prompt against the same model identifier in a dev account, gets the correct answer, and closes the ticket as "cannot reproduce." Two weeks later the same complaint shows up from a different customer. The engineer replays again, in the same dev account, and gets the correct answer again. The team starts blaming a prompt change nobody made.

The model identifier in the request never changed. The string in the response field matched the string in the request field. The eval suite stayed green for six weeks. The model serving production traffic was a different set of weights from the model serving the eval suite, and had been for the entire life of the account — except for the last six weeks, when it became the same set of weights and the team noticed only because a customer noticed first.

The security review said the agent acts "on behalf of" the user. The OAuth token said something else, and the audit log agreed with the token.

A small distinction in language did a lot of architectural work nobody noticed. "On behalf of" is the language a security review reaches for when it wants to capture an arrangement where the agent is a delegate, recognizable as a delegate, and constrained by being a delegate. "Act as" is the runtime behavior when the agent holds a token indistinguishable from the user's own and is therefore the user as far as every downstream system can tell. These two phrases describe completely different threat models. A typical enterprise OAuth integration ships the second one and prices it as the first.