The Process Your Agent Quietly Owns Without Documentation

Six months ago, your team shipped a support agent that handles refunds. There was a one-page Notion doc describing what it should do. Today, the doc still says what it said, but the agent does not. The prompt has 47 edits in its history. Three tools were added — one of them quietly bypasses a finance check that the doc still asserts exists. The model was swapped twice. A retry policy was hardened after an incident nobody wrote up. And when somebody on the data team asks "what are the actual rules for issuing a refund here," the honest answer is: read the system prompt and the tool registry, because that is the spec now.

This is the quiet failure mode of agentic systems in production: the agent's behavior IS the runbook nobody wrote. The prompt got treated as a configuration value — a string in a YAML file, edited by whoever owned the feature, reviewed like a copy change — when it was actually the most authoritative description of a multi-step business process in the company. The org accumulated process logic the way legacy codebases accumulate behavior: through edits, not design. And the people who would historically own that process — a product manager, a compliance lead, an ops director — never realized they had lost the artifact, because there was never a document to lose.

A Prompt Is a Specification, Not a Configuration

The category error is upstream of every other problem. A configuration value tunes a known process: log level, timeout, region. A specification defines a process: what the agent will do, when, under which conditions, with which tools, escalating to whom. Most teams I see treat their system prompt as configuration. It lives next to feature flags. Edits skip the design review that a new code path would trigger. There is no parallel document the prompt is implementing — the prompt is the document.

The literature is starting to catch up to this. Researchers now describe prompts as "software engineering artifacts" with names, versions, explicit inputs, and a pinned set of model settings they were tested against. That framing is useful, but it understates the harder claim: when a prompt encodes a business process, it is not just a software artifact, it is a process artifact — an SOP, a runbook, a policy. The audience for that artifact is not only the model. It is the auditor, the on-call engineer at 3am, the new hire who needs to know how refunds work, and the lawyer who needs to defend the company's decisions.

If you cannot point to a written document that the prompt is faithfully implementing, the prompt is the document. And a document that is also being interpreted at runtime by a probabilistic system is not a document anyone should be comfortable owning by accident.

How the Drift Happens

The drift is invariably gradual, which is what makes it hard to catch. The first version of the agent often does have a parallel doc — a one-pager that says "the support agent issues refunds for orders under $50, escalates anything over $50 to a human, and never refunds twice." For a month or two, prompt and doc agree. Then real life happens.

A customer complains about a missing refund on a $52 order, and an engineer raises the threshold to $75 in the prompt. The doc still says $50. Nobody updates it because the threshold was always going to be tuned, and updating the doc would mean re-circulating it through legal, and nobody has time. Two months later, a different engineer adds a tool that lets the agent issue store credit, because store credit is "basically a refund." The doc never mentioned store credit. Six months later, an incident review concludes the agent should double-confirm refunds over $100, so a new instruction is added: "for any refund exceeding 100 USD, ask the user to restate the order ID before proceeding." Nobody calls that a policy change because nobody calls anything a policy change — they call it a prompt edit.

The pattern shows up in every domain I have looked at:

• Support agents accumulate refund thresholds, blocklists, tone instructions, and escalation rules in the prompt — each one a customer-facing policy commitment.
• Triage agents accumulate severity heuristics, routing rules, paging logic, and on-call notification conditions — each one an operational contract with the team.
• Scheduling agents accumulate working-hours assumptions, buffer rules, VIP overrides, and conflict-resolution heuristics — each one a piece of organizational etiquette nobody wrote down.
• Code-review agents accumulate "the team's style," "the team's risk tolerance," and "the team's escalation rules" — engineering culture, encoded as a string.

In every case, the prompt grew the way a legacy codebase grows: defensively, incrementally, and without anyone deleting anything because nobody can prove what is safe to delete. The new version of the agent ships, the old version is forgotten, and the org now operates on a spec that exists only as runtime behavior.

The Failure Modes Nobody Predicted

The implicit-ownership problem stays invisible right up until it doesn't, and then it tends to surface in one of four ways.

Loading…