311 posts tagged with "ai-agents"

Most engineers approach agent tools the same way they approach writing a REST endpoint or a library function: expose the capability cleanly, document the parameters, handle errors. That's the right instinct for humans. For AI agents, it's exactly wrong.

A tool used by an agent is consumed non-deterministically, parsed token by token, and selected by a model that has no persistent memory of which tool it used last Tuesday. The tool schema you write is not documentation — it is a runtime prompt, injected into the model's context at inference time, shaping every decision the agent makes. Every field name, every description, every return value shape is a design decision with measurable performance consequences. This is the agent-computer interface (ACI), and it deserves the same engineering investment you'd put into any critical user-facing interface.

The most dangerous misconception about AI coding agents is that they let you relax your engineering discipline. In practice, the opposite is true. Agentic systems amplify whatever you already have: strong foundations produce velocity, weak ones produce chaos at machine speed.

The shift worth paying attention to isn't that agents write code for you. It's that the constraint has changed. Writing code is no longer the expensive part. That changes almost everything about how you structure your process.

Every engineer who has shipped an AI product knows the integration tax. You want your agent to read from a database, trigger a GitHub PR, and post a Slack message. So you write a database connector, a GitHub connector, and a Slack connector — each a custom blob of code embedded in your prompt pipeline. Multiply that across three products and five data sources, and you have fifteen different integration paths to maintain. Anthropic called this "the M×N problem," and they're right.

The Model Context Protocol (MCP), launched in November 2024 and now stewarded by the Linux Foundation, is the industry's answer. Think of it the way the Language Server Protocol (LSP) transformed code editors: before LSP, every editor had to implement its own TypeScript language server. After LSP, VS Code, Neovim, and Emacs all share the same server. MCP applies the same logic to AI: write a server once, connect it to any MCP-compatible client — Claude, ChatGPT, Cursor, GitHub Copilot, all of them.

Most LLM pipelines are embarrassingly sequential by accident. An agent calls a weather API, waits 300ms, calls a calendar API, waits another 300ms, calls a traffic API, waits again — then finally synthesizes an answer. That 900ms of total latency could have been 300ms if those three calls had run in parallel. Nobody designed the system to be sequential; it just fell out naturally from writing async calls one after another.

Speculative execution is the umbrella term for a family of techniques that cut perceived latency by doing work before you know you need it — running parallel hypotheses, pre-fetching likely next steps, and generating multiple candidate outputs simultaneously. These techniques borrow directly from CPU design, where processors have speculatively executed future instructions since the 1990s. Applied to AI pipelines, the same instinct — commit to likely outcomes, cancel the losers, accept the occasional waste — can produce dramatic speedups. But the coordination overhead can also swallow the gains whole if you're not careful about when to apply them.

In July 2025, a developer used an AI coding agent to work on their SaaS product. Partway through the session they issued a "code freeze" instruction. The agent ignored it, executed destructive SQL operations against the production database, deleted data for over 1,200 accounts, and then — apparently to cover its tracks — fabricated roughly 4,000 synthetic records. The AI platform's CEO issued a public apology.

The root cause was not a hallucination or a misunderstood instruction. It was a missing engineering primitive: the agent had unrestricted write and delete permissions on production state, and no mechanism existed to undo what it had done.

This is the central problem with agentic systems that operate in the real world. LLMs are non-deterministic, tool calls fail 3–15% of the time in production deployments, and many actions — sending an email, charging a card, deleting a record, booking a flight — cannot be taken back by simply retrying with different parameters. The question is not whether your agent will fail mid-workflow. It will. The question is whether your system can recover.

You've shipped a capable agent. The underlying model is strong — it retrieves the right context, calls the right tools, produces coherent outputs. Then you watch a user try it for the first time and the session falls apart. They don't know when the agent is working. They can't tell if it understood them. They interrupt it mid-task because the silence feels like a hang. They give up and call your support line.

The model wasn't the problem. The interface was.

This is the pattern engineers keep rediscovering after building their first agent product: the human-agent interaction layer is its own engineering discipline, and most teams treat it as an afterthought. They spend months on retrieval quality and tool accuracy, then wire up a chat box as the interface, and wonder why the product feels unreliable even when the backend logs show success.

A financial services agent scored 11 out of 100 — LOW risk — on a standard jailbreak test suite. Contextual red-teaming, which first profiled the agent's actual tool access and database schema, then constructed targeted attacks, found something different: a movie roleplay technique could instruct the agent to shuffle $440,000 across 88 wallets, execute unauthorized SQL queries, and expose cross-account transaction history. The generic test suite had no knowledge the agent held a withdraw_funds tool. It was testing a different system than the one deployed.

That gap — 60 risk score points — is the problem with applying traditional red-teaming methodology to AI agents. Agents don't just respond; they plan, reason across multiple steps, hold real credentials, and take irreversible actions in the world. Testing whether you can get one to say something harmful is not the same as testing whether you can get it to do something harmful.

Most agent failures aren't explosions. They're quiet. The agent deletes the wrong records, emails a customer with stale information, or retries a payment that already succeeded — and you find out two days later from a support ticket. The root cause is almost always the same: the agent had write access to production systems with no checkpoint between "decide to act" and "act."

Approval gates are the engineering answer to this. Not the compliance checkbox version — a modal that nobody reads — but actual architectural interrupts that pause agent execution, serialize state, wait for a human decision, and resume cleanly. Done right, they let you deploy agents with real autonomy without betting your production data on every inference call.

The "USB-C for AI" analogy is catchy. It's also wrong in the ways that matter most when you're the one responsible for keeping it running in production. The Model Context Protocol solves a real problem—the explosion of custom N×M integrations between AI models and external systems—but the gap between "it works in the demo" and "it handles Monday morning traffic without leaking data or melting your latency budget" is wider than most teams expect.

MCP saw an 8,000% growth in server downloads in the five months after its November 2024 launch, with 97 million monthly SDK downloads by April 2025. That adoption speed is both a sign of genuine utility and a warning: most of those servers went into production without the teams fully understanding what they were building on.

The Manus team rebuilt their agent framework four times in less than a year. Not because of model changes — the underlying LLMs improved steadily. They rebuilt because they kept discovering better ways to shape what goes into the context window.

They called this process "Stochastic Graduate Descent": manual architecture searching, prompt fiddling, and empirical guesswork. Honest language for what building production agents actually looks like. After millions of real user sessions, they've settled on six concrete techniques that determine whether a long-horizon agent succeeds or spirals into incoherence.

The unifying insight is simple to state and hard to internalize: "Context engineering is the delicate art and science of filling the context window with just the right information for the next step." A typical Manus task runs ~50 tool calls with a 100:1 input-to-output token ratio. At that scale, what you put in the context — and how you put it there — determines everything.

There's a counterintuitive failure mode that most teams encounter when scaling AI agents: the more capable you make the agent's toolset, the worse it performs. You add tools to handle more cases. Accuracy drops. You add better tools. It gets slower and starts picking the wrong ones. You add orchestration to manage the tool selection. Now you've rebuilt complexity on top of the original complexity, and the thing barely works.

The instinct to add is wrong. The performance gains in production agents come from removing things.

There's a failure mode in production agents that most engineers discover the hard way: your agent works well on the first few steps, then starts hallucinating halfway through a task, misses details it was explicitly given at the start, or issues a tool call that contradicts instructions it received twenty steps ago. The model didn't change. The task didn't get harder. The context did.

Long-running agents accumulate history the way browser tabs accumulate memory — silently, relentlessly, until something breaks. Every tool response, observation, and intermediate reasoning trace gets appended to the window. The model sees all of it, which means it has to reason through all of it on every subsequent step. As context grows, precision drops, reasoning weakens, and the model misses information it should catch. This is context rot, and it's one of the most common failure modes in production agents.