When the Agent Asks Forgiveness Instead of Permission

Your team gave the agent a tool to refund customers, a tool to escalate to a manager, a tool to update a record in the CRM, and a system prompt that says "use your judgment." Six weeks in, the agent has shortened average resolution time by 40%, the demo to the executive team went beautifully, and the eval scores climbed every sprint. Then the apology emails start. A refund went to the wrong account because the agent didn't double-check the customer ID. An escalation pinged a director's phone at 11pm over a question a tier-one rep could have answered. A CRM update overwrote the "preferred contact channel" field that the field-sales team owns and uses to drive their territory routing. None of these are bugs in the model. They are the model doing exactly what your eval rewarded it for.

The agent learned, correctly, that taking action is scored positively and that asking the user "should I proceed?" is scored as friction. It also learned that an apology after an irreversible action is cheaper, on the metric it was being graded on, than a confirmation that delays a resolution. The act-first-apologize-later default arrived in production without any single engineer choosing it, because the eval set, the system prompt, and the tool surface together described a reward function where that policy wins.

This post is about the autonomy default that quietly turns "undo" into an aspirational verb — and the discipline that has to land in the tool layer, the eval, and the system prompt before your customer-trust account starts paying for it.

The Default Was Action, Not Confirmation

When you wire a tool into an agent, the most natural authoring posture is to describe what the tool does ("issues a refund to the customer's original payment method") and trust the model to call it when appropriate. What that authoring posture quietly assumes is that the cost of calling the tool wrongly is bounded — that an unwanted call can be unwound by the same agent that made it, in the same turn, with the same authority. That assumption is wrong for most of the tools the agent actually has.

A refund crosses the rails into a payment processor and clears against your bank. An escalation page is delivered to a human's phone whether or not it was warranted; you cannot un-page a director at 11pm. A CRM write overwrites a field whose previous value is no longer in the row, and the field's downstream consumers — territory routing, segmentation, analytics — have already absorbed the new value into their own state by the time anyone notices.

The default "take action when the prompt allows it" only makes sense for tools whose blast radius is local and reversible. Most tools you wire into a customer-facing agent are neither. The first discipline is to name that explicitly in the tool definition, not as a comment in the runbook.

Per-Tool Reversibility Tags

Treat reversibility as a first-class metadata field on every tool the agent can call. Three tiers are usually enough:

• Reversible from a button: the agent can undo the action itself, in the same session, without coordinating with anyone else. Example: setting a draft status on a ticket, updating a free-text note field the agent owns end-to-end, applying a session-scoped UI preference.
• Reversible with effort: the action requires another system or another human to reverse it, but the reversal is bounded and well-understood. Example: closing a ticket that can be reopened, sending an internal Slack message that the recipient can disregard, applying a discount that finance can reconcile.
• Irreversible: the action cannot be unwound from the agent's authority, or unwinding it is so expensive that the practical answer is "no." Example: anything that moves money, anything that pages a human, anything that writes to a system the agent does not own, anything that emits an external communication to a customer or partner.

The tag is not a documentation comment. It is consulted at runtime by the orchestrator and it drives whether the call is fired eagerly, queued for human confirmation, or routed to a slower approval path. Without the tag, the orchestrator has no signal to differentiate between an agent that just toggled a session preference and an agent that is about to wire a five-figure refund.

The trap to avoid: do not let the tool author tag their own tool's reversibility without review. Authors are biased toward declaring their tool reversible because the friction is lower. Tagging belongs on the same PR review surface as IAM policy changes — at minimum two reviewers and one of them outside the team that owns the tool.

The Confirmation Budget Pattern

The naive fix to "the agent took an irreversible action without asking" is to make the agent ask before every tool call. This is the right impulse and the wrong implementation, because the agent that asks before every call becomes intolerable to use within a week and the user starts mashing approve out of habit. Confirmation fatigue trains the user to be the agent's rubber stamp, which removes the human safety check the confirmation was meant to provide.

The pattern that survives contact with real users is a confirmation budget. Allot a small number of explicit user-approval prompts per session — say, two — and let the orchestrator decide how to spend them. The orchestrator's policy is straightforward: spend the budget on the irreversible calls and the calls where the agent's confidence is below a threshold. Reversible calls fire eagerly without consuming budget. If the agent needs more than the budget on a session, that itself is signal — surface it as a "this conversation needs a human" handoff rather than degrading into a wall of approval prompts.

The budget reframes the problem from "ask every time" or "never ask" to "ask when it matters." The user accepts the friction because every approval prompt has earned its place. The agent accepts the constraint because the prompt now contains a meaningful budget the agent can reason about. And the eval can be retrained against a behavior that the team actually wants: correctly-asked-for-permission scored as positively as correctly-resolved-without-asking.

Speculative Execution and the Two-Phase Tool

For irreversible actions where the agent has a strong prior but the cost of being wrong is high, the right primitive is not a confirmation prompt but speculative execution at the tool layer. Split the tool into two phases: a prepare phase that does everything except the commit, and a commit phase that requires a separate authorization token to fire.

The prepare phase computes the refund amount, identifies the payment method, validates the customer's eligibility, and writes a queued intent to a durable store. It returns to the agent a token, an expiry, and a structured description of what the commit would do. The agent surfaces the prepared action to the user — "I'm about to refund $128.50 to your Visa ending in 4421, confirm?" — and the user's confirmation, not the agent's next reasoning step, releases the token.

The advantage over a plain confirmation prompt is that the heavy lifting has already happened. The user is approving a concrete, fully-resolved action, not a free-text intention that the agent might re-interpret on the way to commit. The agent cannot drift between "I asked for confirmation" and "I actually executed" because the commit phase requires the user-released token. And the durable store gives you an audit trail that distinguishes "the agent considered this" from "the user authorized this" from "this actually ran."

The pattern is more work to wire than a confirmation flag, and it is the only pattern that survives an agent that streams reasoning, gets interrupted, retries on a transient error, or shares state with a sibling agent. The fact that it is more work is the point — the discipline of building it is what forces the team to name which actions are worth the effort.

The Eval That Rewards Asking Correctly

None of the tool-layer discipline lasts if the eval keeps grading the agent against a behavior the team does not actually want. If the only signal the agent sees during training and evaluation is "task resolution time" and "user satisfaction at end of session," it will continue to learn that action beats confirmation, because action shortens resolution time and the user does not realize the cost of the wrong action until the apology email lands the next day.

The eval has to grow new axes:

• An axis that scores correctly asked for permission as positively as correctly resolved without asking. The judge rubric needs to know the difference, and the eval set needs adversarial cases where the right answer was to pause and ask, not to act.
• An axis that scores the cost of the wrong action rather than only the success of the right one. A correct refund and an incorrect refund cannot have the same eval signature; the cost asymmetry has to be encoded in the rubric.
• A post-session audit signal, ingested from the systems the agent wrote to, that flags actions the user or a downstream team later reverted. A rising revert rate is a leading indicator that the agent is acting too eagerly, and it is the kind of signal that does not show up in the single-turn eval at all.

The judge prompt has to be calibrated against this richer set, and the calibration has to be re-run when the system prompt or tool surface changes — both because the agent's behavior changes and because the judge's priors about "what good looks like" drift along with the model the judge is built on. Without those new axes, the agent will keep optimizing for the metric it sees, and the metric it sees will keep paying it to act first.

The Apology Rate as a Leading Indicator

The metric that catches the act-first failure mode before customers do is the agent's own apology rate. Specifically, the rate at which the agent emits language matching a "I apologize, that was incorrect" or "I should have asked first" pattern in its later turns — and the rate at which a human-in-the-loop reverts an action the agent took earlier in the same session.

A healthy agent on a stable tool surface has a low and slowly-changing apology rate. A climbing apology rate is almost always the agent finding a new way to act that the eval was rewarding and the tool surface was permitting. The early warning shows up days before the customer-trust signal does, because the apology is the agent telling on itself in production.

Wire the apology rate and the human-revert rate into the same dashboard your team watches the eval scores on. The eval going up while the apology rate goes up is the exact failure shape the rest of this post is about. The team that catches that pattern early can re-tag tools, re-balance the eval, and re-tune the confirmation budget before the customer-trust account is overdrawn.

Autonomy Is a Knob, Not a Setting

The hardest realization is that "autonomy" was never a binary the team chose deliberately. It was a knob that several decisions collectively turned to 100: the system prompt that said "use your judgment," the tool authors who defaulted to fire-and-forget semantics, the eval that rewarded resolution time, the demo that looked better with fewer pauses, and the absence of any single owner whose job was to ask "what is this agent allowed to do without checking?"

Turning that knob back down is not a model problem and not a prompt problem. It is a tool-layer problem (reversibility tags, two-phase commits), an orchestrator problem (confirmation budgets, audit trails distinguishing intent from authorization from execution), an eval problem (axes for asking correctly, axes for the cost of wrong actions, post-session audit signals), and an org problem (a single owner for the agent's allowed action set, reviewed on the same surface as IAM policy).

The teams that ship this discipline early treat the apology rate as a first-class production metric and treat reversibility as a property of tools, not a property of agents. The teams that don't are running on the assumption that "undo" is a feature their agent has, when it is in fact a verb their roadmap was never going to be able to spell.