44 posts tagged with "agent-architecture"

A senior engineer ships a four-word edit to the planner prompt — "if uncertain, ask first." The planner's own eval set, which grades whether plans are reasonable, moves up by half a point. They merge. Two weeks later, the verifier's eval shows a three-point pass-rate regression and nobody can repro it. The root cause turns out to be that the planner now asks more clarifying questions, the executor receives shorter task descriptions on the second turn, the verifier's rubric was implicitly tuned against the previous executor's longer outputs, and an edit nobody flagged as risky has shifted three downstream distributions at once.

This is what happens when you treat the prompts inside an agent as a flat folder of files instead of as a graph with edges. The prompts have owners. The edges between them have nobody.

When a single-pass agent stops being good enough, the default move is to wrap it in a self-critique loop. Generate, critique, revise, repeat. Most teams I talk to assume the eval lift will be roughly linear with the number of revision rounds and stop there. The numbers rarely cooperate. By the third round of self-critique, accuracy is up two or three points and token cost is up 3–4x, and the failure modes that didn't get caught in round one mostly don't get caught in round three either — because the same context that produced the wrong answer is the one being asked to spot the wrongness.

A different shape works better and costs less: let the first pass be wasteful exploration, throw it away, and run a second pass from a clean context with just the lessons learned. Call it the second-draft pattern, or explore-then-commit. The first draft is permitted to be sloppy, to take dead ends, to dump scratch artifacts, to chase hypotheses that turn out to be wrong. The second draft is constrained — it gets the distilled findings and produces a clean execution. On the kinds of tasks where self-critique is tempting (multi-step reasoning, code that touches several files, research syntheses), this two-pass shape often beats n-of-k self-critique on both quality and cost.

A SQL tool ships rows as they come off the wire. The agent calls it expecting a result. The harness, written a year earlier when every tool was request-response, dutifully buffers the whole stream into a single string before invoking the model. Forty seconds later, the buffer is 200 KB, the context window is half-eaten, and the agent is reasoning about row 47,000 of a query it could have stopped at row 30. Nobody designed this failure — it falls out of treating "the tool returned" as the only event the planner reacts to.

The shift to streaming tools is happening below the planner's awareness. SQL engines emit progressive result sets. Document fetchers yield pages. Search APIs return hits in batches as relevance scores stabilize. MCP's Streamable HTTP transport, the 2025-03-26 spec replacement for HTTP+SSE, makes incremental responses a first-class transport mode rather than an exotic capability. The wire is ready. The planners on top of it are not.

There's an uncomfortable truth about LLMs that doesn't get discussed enough in product reviews: the property that makes them useful is identical to the property that makes them exploitable. An LLM that obediently follows instructions — any instructions, from any source, delivered in any format — will follow malicious instructions with the same cheerful compliance it applies to legitimate ones. The model cannot tell the difference.

This isn't a bug that will be patched away. It's an architectural reality. And as these systems take on more agentic roles — reading emails, browsing the web, executing code, calling APIs — the exposure surface grows in ways that most engineering teams haven't mapped.

Most hallucination research focuses on what comes out of a single model call. That framing misses the scarier problem: what happens in a four-stage pipeline where each stage unconditionally trusts the previous output. A single hallucinated fact in Stage 1 doesn't just persist—it becomes the load-bearing premise for every subsequent inference. By Stage 4, the pipeline delivers a confident, internally coherent answer that happens to be entirely wrong.

This isn't a capability problem that better models will solve. It's a systems architecture problem, and it requires a systems-level fix.

An agent running in production once received the instruction "clean up the test data" and executed a DROP TABLE command against a production database. The tool call succeeded. The audit log showed a perfectly structured JSON payload. The agent had done exactly what it was asked — just not what anyone meant. This isn't a story about prompt injection. It's a story about an architectural choice: the team had given their agent the ability to generate and execute arbitrary code, and they had underestimated what that actually means at runtime.

The choice between function calling and code generation as the action layer for AI agents is one of the most consequential decisions in agent architecture, and almost nobody benchmarks it directly. Papers measure accuracy on task completion; they rarely measure the failure modes that matter in production — silent semantic errors, irreversible side effects, security exposure surface, and debugging cost when something goes wrong.

When a PlanCompiler-style agent is benchmarked against direct LLM-to-code generation on 300 stratified multi-step tasks, the structured approach achieves 92.67% success at $0.00128 per task. The direct approach — where the LLM decides actions step-by-step in a free-form loop — achieves 62% success at $0.0106 per task. That is 50% more accurate at one-eighth the cost.

The difference isn't model capability. Both approaches use the same model. The difference is architecture: one separates plan generation from plan execution; the other conflates them.

The most user-hostile button in modern AI products is also the most necessary one. Somewhere around message forty, the agent has latched onto a wrong assumption, the tone has drifted, and every new turn is making the answer worse instead of better. The user knows the right move: clear the slate and start again. They reach for "New Chat" — and watch the half-finished plan, the four documents they drafted, and the configured prompts they spent twenty minutes shaping vanish along with the poisoned history.

So they stop using the reset button. They open a second tab, copy-paste their artifacts across by hand, and keep the broken conversation alive as a graveyard they're afraid to close. That ritual — manual copy-paste as a workaround for a button that should have done the right thing — is the clearest signal a chat product can give that its data model is wrong.

The first incident is rarely an outage. It's a Slack message from someone in customer success: "Hey, are we OK? Five customers in the last hour escalated tickets that have been sitting in 'awaiting review' for over a day." You check the model latency dashboard. Green. You check the agent's success rate. Green. You check the cost-per-call graph. Healthy. Everything you instrumented is fine. The thing that's broken is a queue your monitoring stack doesn't know exists, staffed by people whose calendars your capacity planner doesn't read, governed by an SLA that nobody has ever written down.

That queue is your human-in-the-loop escalation path. You added it three months ago "for safety" — the agent would defer to a human reviewer on the small fraction of cases where its confidence was low or the action was high-stakes. At launch it caught maybe a dozen items a day. The ops team handled them between other tasks. It was a backstop, not a system. Today it's processing thousands of items, the median time-to-resolution has tripled, and the customers waiting in line are quietly churning. The HITL path didn't fail. It just stopped being treated like production.

A team I talked to last quarter had every dashboard green. Tool error rate flat at 0.3%. End-to-end success at 98%. SLO budget barely touched. They were also burning four times their projected token spend, and nobody could explain it. When they finally instrumented retry depth per trace, the picture inverted: the median successful request was making 2.7 tool calls instead of the 1.0 the architecture diagram promised. The agent was not failing. It was failing and recovering, over and over, inside the same span, and the success rate metric had no way to tell them.

This is the part of agentic reliability that the old reliability vocabulary cannot reach. MTBF — mean time between failures — assumes failures are punctuated, observable events you can count between. You measure the gap, you compute the mean, you alert when the gap shrinks. It worked for hard drives, networks, deterministic services. It does not work for systems that retry, reroute, fall back, and recover silently inside a single user-visible operation.

Look at the loop primitives most agent frameworks ship: continue, return, retry, and a step budget that hard-stops the run. Notice what is missing. There is a path that says "the work succeeded," a path that says "the model wants to keep going," and a path that says "we ran out of money or patience and shot the loop in the head." There is no first-class path that says "the plan I am executing is wrong, and I want to throw it away and start a different one." The abandon primitive — an explicit, structured way for the planner to declare its current trajectory hopeless — is the missing verb in the agent loop's grammar, and its absence is responsible for a category of failures that are usually misdiagnosed as "the model needs more reasoning."

A planner three steps into a doomed branch keeps refining the same wrong plan because the loop's only exits are succeed, retry the last step, or hit the budget. None of those are "give up on the strategy and try a different one." So the agent does what the loop allows: it edits its plan in place, calls one more tool, asks for one more clarification, and burns through its step budget converging on a non-solution. When the wall finally hits, the user sees a polite failure message that is not an answer to their question. The cost of those wasted steps is real — production data suggests 5–10% of token spend on agent systems goes into retries that produce nothing usable, and that figure is dominated by long doomed branches, not isolated tool errors.

Your agent just spent ninety seconds and a few dollars to change a three-line function. Before the edit landed, it listed two directories, opened the test file, ran a grep for callers, read the config module, checked the CI workflow, and pulled up a type definition it never used. The diff it produced was four lines. The trace that produced it was forty-three tool calls.

This is first-touch tool burn: the pattern where an agent, handed a well-scoped task, behaves as if every request is a research problem. The exploration happens first and it happens hard — sixty to eighty percent of the token budget spent on listing, grepping, and reading before a single character is written to a file. Teams discover this the first time they look at a trace and realize the agent did the equivalent of a two-hour onboarding for a two-minute task.

The behavior isn't a bug in any specific model. It's the predictable output of how these systems were trained and evaluated, colliding with a production environment that measures something training never did: whether the work was cheap enough to bother doing at all.