Your Agent Has Two Release Pipelines, Not One

A team I worked with shipped a "small prompt tweak" on a Wednesday afternoon. The same PR also added one new tool to the agent's registry — a convenience wrapper around an internal admin API that the prompt would now occasionally invoke. The eval suite passed. The canary looked clean. By Thursday morning a customer's billing record had been mutated by an agent acting on a prompt-injected support ticket, the audit trail showed the admin tool firing exactly as designed, and the on-call engineer's first instinct — roll back the prompt — did nothing useful, because the credential had already been used and the row had already been written.

The post-mortem framed it as a security review failure. It wasn't. It was a release-pipeline failure. The team had shipped two completely different asset classes — a behavioral nudge to the model and a new authority granted to the agent — through the same review, the same gate, and the same rollback story, as if they were the same kind of change. They aren't. And once you see them as two pipelines, most "agent governance" debates become much less mysterious.

Two asset classes that look identical on disk

Open the repo of any non-trivial agent and you'll see two folders that, structurally, look indistinguishable. Both are versioned text. Both are reviewed in pull requests. Both deploy with the same CI job. But they sit on opposite sides of a fault line that networking architects identified forty years ago and that we keep re-discovering badly.

The data plane of an agent is the model's behavioral shape: system prompts, instructions, few-shot examples, output schemas, persona descriptions, refusal guidelines, the whole literary corpus that nudges the model toward one distribution of outputs over another. Edits here change what the model says.

The control plane is the agent's reachable action space: the tool registry, tool schemas, capability scopes, auth boundaries, rate-limit budgets, the list of MCP servers the agent can talk to, the policies that gate which tools fire under which conditions. Edits here change what the agent can do.

These are not two flavors of the same thing. They have different blast radii, different rollback semantics, different reviewers, different audit obligations, and different failure modes — and treating them as one pipeline is the architectural mistake at the root of most "the agent did something wild in production" incidents.

The asymmetry of failure

A bad data plane change produces a quality regression. The agent gets too verbose, refuses requests it shouldn't, hallucinates a field name, repeats itself, switches tone mid-conversation. You see it in eval deltas, user complaints, satisfaction scores. You roll the prompt back to the previous version and the regression goes with it. The blast radius is bounded by your monitoring window, and the recovery is mechanical: every conversation after the rollback uses the new behavior, and the only residue is a few hours of mediocre transcripts.

A bad control plane change produces an authority regression. The agent didn't say something you wished it hadn't — it did something you wished it hadn't. Sent the email. Cancelled the subscription. Wrote the row. Granted the access token. Posted to the channel. The model can roll back; the side-effects can't. Your "rollback" is now an incident response that involves credential rotation, communications to affected users, possibly a regulator phone call, and a lawyer asking pointed questions about what your audit log actually captured. You don't recover from a bad tool registration the way you recover from a bad prompt. You contain it, and you live with the residue.

OWASP's LLM06 guidance frames this as "excessive agency" and explicitly calls it a fundamental design tension rather than a configurable knob. That language matters. It says the right control isn't a setting on a dashboard — it's an architectural choice about which side of the data/control divide a change lives on, and what review it gets accordingly.

Why one pipeline is the default

Almost every team starts with one pipeline because, on day one, the asset classes haven't differentiated yet. The "agent" is a single Python file with the system prompt at the top, a TOOLS list near the bottom, and a main() function that wires them together. One repo, one CI, one reviewer. Treating it as two release surfaces would be ceremony for ceremony's sake.

The trouble is that the two asset classes start to grow at very different rates and into very different shapes — and the pipeline that was correctly fused on day one stays fused long past the point where the fusion does any work. The system prompt grows from 30 lines to 300, with regression tests, eval suites, behavioral diffs, A/B harnesses, and a versioning story. The tool registry grows from three functions to thirty, with auth scopes, rate-limit budgets, blast-radius classifications, and the long quiet drift of "we added one more capability for that one Q3 use case and never re-audited the whole list."

A year in, you have one pipeline doing two jobs poorly. The prompt edits go through eval-delta review (good — that's what they need) and the tool additions go through eval-delta review (bad — that's not what they need; what they need is capability-scope review, taint analysis, blast-radius classification, and a security signature). And because the same PR template, the same reviewers, and the same merge button cover both, the engineer adding a tool gets the prompt-engineer's gate by accident. The review never asks the right questions. The eval suite measures behavior, not authority — and the new tool, by definition, expands authority.

What separation actually looks like

Splitting the pipelines doesn't mean two repos and two CI systems (though some teams arrive there). It means that data-plane and control-plane changes pass through different gates even when they ride in the same PR.

A data-plane change is gated on behavioral diff. Did the eval suite move? Did the new prompt regress refusal accuracy, latency, formatting? Did the offline replay against last week's traffic produce different outputs, and are those differences improvements? The reviewer is a prompt engineer or a domain expert. The rollback is a git revert. The deploy timing is fast because the consequences are recoverable.

A control-plane change is gated on capability-scope diff. What can the agent do today that it could not do yesterday? Which credentials does the new tool need? What's the blast radius if it fires under a manipulated input — a no-op, a reversible write, an irreversible side-effect, a privileged action? Does it compose with existing tools to create new dangerous paths the original threat model didn't enumerate? The reviewer is a security engineer or someone who owns the auth boundary. The rollback may not actually undo anything (the email got sent), so the gate is a forward gate — don't ship it until you've thought about what happens when it misfires — rather than a roll-it-back-if-bad gate.

Concretely, the discipline that follows from separating the two:

• Independent registries for prompts and tools, each with its own owners, change-log, and review CODEOWNERS. The same engineer can edit both, but the gates are different.
• Different signoff criteria. Prompts ship on green eval deltas. Tools ship on capability-scope sign-off, regardless of whether the eval suite happened to pass.
• Decoupled deploy timing. A prompt fix shouldn't wait on a security review for a tool change that happened to land in the same sprint. A tool revocation shouldn't wait on a prompt freeze. They need to be independently shippable.
• Separate observability. One dashboard for behavioral drift (refusal rates, output length, format compliance, eval-suite scores). Another for authority drift (which tools fired in the last 24 hours, against which scopes, on whose behalf, with what input provenance). Incident review pulls from both, but the dashboards are owned and read by different teams.
• Separate audit logs. The behavioral log captures what the model said. The authority log captures what the agent did. Compliance reviews the second one; product reviews the first. Mixing them turns an audit into a needle hunt.

The separation isn't about adding bureaucracy. It's about routing each kind of change to the gate that can actually catch what's wrong with it.

The org failure mode

The cleanest technical separation in the world won't survive an org chart that says "the prompt engineer also owns the tool registry because it's all just config." That's the failure mode I've watched most often: one engineer holds both, neither half gets the right kind of review, and the tool registry quietly accumulates capabilities nobody auditioned. Six months in, the agent can do twelve things it could do at launch plus a quiet thirteenth that someone added for a customer escalation in March, and the tooling team has no idea because that engineer reads them as "their" stack.

The fix is uncomfortable but boring: the control plane needs a different owner than the data plane, even if the same person commits to both. "Owner" here means accountable for the gate, not accountable for the code. The prompt engineer can write the tool, but a security-track reviewer signs off on the capability-scope diff before it merges. This costs you a few hours of latency on tool changes; it saves you the incident where you discover at 3am that a tool you forgot you had has been quietly used by the agent for a month.

The 2026 industry data is bleak on this point: a state-of-agent-security report this year found that 70% of enterprises have agents in production while only 14% have full security approval for those agents, and over half of deployed agents operate without authority logging at all. That's not a security-team failure. It's a release-pipeline failure — the org never built a control-plane review surface, so authority changes route through the data-plane gate by default and the gate doesn't know to look.

The networking analogy is doing real work

It's worth saying plainly why borrowing the data-plane / control-plane vocabulary from networking is more than a cute metaphor. In a router, the design separation exists because the failure modes scale differently: a forwarding bug stalls one packet, a routing bug black-holes a whole prefix. The two are optimized differently — the forwarding path for speed and regularity, the control path for customizability and exception handling — precisely because conflating them gives you neither.

Agents have the same shape. A behavioral bug stalls one conversation; an authority bug stalls a whole class of side effects. The behavioral plane wants fast iteration, eval gates, A/B harnesses, and lightweight rollback. The authority plane wants slow change, explicit scoping, security gates, and forward containment because rollback isn't always available. Trying to optimize one pipeline for both is the same mistake routers stopped making in the 1990s.

The implication for any team building agents past the toy stage: stop asking "what's our agent deployment process." Ask "what are our two agent deployment processes, and which gate does each kind of change pass through." If you can't answer that with two different reviewer lists, two different signoff criteria, and two different observability surfaces, your release pipeline is fused — and the next incident is going to teach you which side of the fault line you weren't watching.