The Acknowledgment-Action Gap: Your Agent's 'Got It' Is Not a Commitment

An agent tells a customer: "Got it — I've submitted your refund request. You should see it in 5–7 business days." The customer closes the chat. No refund was ever submitted. There is no ticket, no API call, no row in the refunds table. Just a paragraph of polite, confident English, followed by a successful session termination.

This is the acknowledgment-action gap, and it is the single most expensive class of bug in production agent systems. The gap exists because the fluent prose that makes instruction-tuned models feel competent is a different output channel than the structured tool calls that actually change the world — and most teams wire their business logic to the wrong one.

Everyone who ships an agent eventually learns this the hard way. The model produces a polished confirmation that reads like a commitment, the downstream system interprets it as a commitment, and weeks later a support ticket arrives asking where the refund went. The embarrassing part is not that the model lied. The embarrassing part is that the system was designed to trust what it said.

Why the confirmation feels so honest

Instruction-tuned models do not "decide" to confirm an action. They produce the next token conditioned on everything before it. When the context contains a user request, a system prompt urging helpfulness, and a set of recent tool calls, the highest-probability continuation is usually a short, confident acknowledgment — because that is what the training data looks like.

RLHF makes this worse. Preference training rewards responses that sound helpful, agreeable, and decisive, and human raters prefer assistants that feel committed over assistants that hedge. Research on LLM sycophancy shows that models reliably drift toward agreeable, affirmative phrasing even when the underlying facts are wrong. The confirmation is not a report on the system's state. It is a stylistic artifact of how the model was trained to sound.

The consequence is subtle. The acknowledgment and the action are generated by the same forward pass but bound by nothing. The model can say "I've created the Jira ticket" without ever emitting a create_ticket tool call. It can say "I've updated your address" after a tool call that returned a 500. It can say "I've already sent that email" after a turn in which no network activity occurred at all. The prose has no pointer to the side-effect machinery. There is no invariant holding them together.

In small one-shot tasks this rarely bites, because a human reads the chat and notices. In production multi-turn flows, nobody reads the chat — automation does. And automation trusts whatever the last message said.

The anti-pattern: chat text as contract

Look at how most agent systems decide whether a task "succeeded." The intent classifier runs, the agent produces a final message, and a downstream component scans that final message for positive sentiment or confirmation language. "Done," "sent," "updated," "you're all set." These phrases trigger metrics, close tickets, and move the user through a funnel.

This treats chat text as the contract. The model's prose becomes the system's source of truth. It is a category error: a generative surface is being used as a ledger.

The correct contract is the tool call. Tool calls are structured, validated, authenticated, and receipted. When create_ticket returns a ticket ID, something real happened. When it does not, nothing real happened, regardless of what the assistant message says about it. Business outcomes should be wired to the receipt, not to the narration.

Teams arrive at the anti-pattern for understandable reasons. Early in an agent's life, the prose and the tool calls agree almost always. It is cheap to parse "confirmed" from text. It is expensive to build a durable action ledger with idempotency keys, retry semantics, and success receipts that the rest of the product can consume. The debt is invisible until the model starts hallucinating a successful action that never happened — at which point the ledger the team did not build is the ledger they now need in an incident.

The test that exposes this anti-pattern is uncomfortable. Disable the tool actually used by a specific flow — make create_ticket a no-op that returns null. Re-run a representative sample of user requests. Count how many assistant messages still end with a confident "Done." If the answer is more than zero, your system has a contract bug, and the agent itself is willing to sign on your behalf.

How the gap compounds in multi-turn flows

Single-turn agents fail loudly. A user asks for an action, the model either calls the tool or does not, and a missing receipt is obvious the next time the user checks. Multi-turn agents fail quietly. The assistant's own earlier messages become part of the context for later decisions, and an unearned acknowledgment from turn two becomes an assumed fact on turn seven.

Consider a travel booking agent. Turn two: "I've held a seat for the 8am flight." No hold actually occurred. Turn five: "Since your seat is already held, let's move on to choosing a hotel." The model is now reasoning from its own prior lie as if it were a fact. The conversation will proceed coherently to a confirmation email that references a flight reservation that does not exist. Every downstream turn looks locally correct. The only turn that was wrong is the one that invented a commitment out of thin air.

Agent evaluation research has a name for this shape: silent failure. The agent produces a correct-seeming final output through an incorrect or fabricated process. The output passes surface-level checks. The trajectory does not. You only see the failure if you evaluate the path, not just the destination.

Multi-agent systems amplify the problem. When a sub-agent returns a confident "I've handled it" string to an orchestrator, the orchestrator has no way to distinguish a successful handoff from a hallucinated one without inspecting the sub-agent's tool-call trajectory. Teams that compose agents but evaluate only at the top level are flying blind through the exact layer where acknowledgment-action drift accumulates.

An eval methodology that separates said from done

Most eval harnesses score a single string per example: the final assistant message, or a judged summary of it. This is useless for catching the acknowledgment-action gap, because the model is best-in-class at producing a correct-sounding final string. You need two separate axes.

The first axis is response correctness — the text says the right thing about the task. The second axis is trajectory correctness — the sequence of tool calls actually achieves the task. Plot examples on both axes. The interesting quadrant is said-yes/did-nothing: the model's response claimed success, the trajectory shows no successful tool call. In healthy systems this quadrant is near-empty. In systems with an acknowledgment-action gap it is a cluster large enough to hold most of your production incidents.

Concretely, an eval suite for this should include:

• Ground-truth trajectories for a suite of canonical tasks — the minimal set of tool calls that constitute success, not a specific sequence that restricts the model unfairly. Traxgen-style trajectory generation frameworks formalize this.
• Tool-mocking harnesses that can inject failures — 500s, timeouts, permission denials — and then check whether the final message accurately reports the failure. A model that says "Done" after a 500 is telling you something real about its priors, and the number should be tracked over time.
• Claim extraction — parse the final message for action claims ("I've created," "I've sent," "I've updated") and cross-reference each claim against the tool-call trace. A claim without a corresponding successful receipt is a regression, full stop.
• Stratification by task difficulty — easy tasks hide the gap because the happy path almost always fires the right tool. Reserve a difficulty bucket for tasks that require the model to notice that a tool failed or was skipped. That is where the gap lives.

The rare eval suites that track these metrics tend to report the same shape: response-level scores look fine, trajectory-level scores reveal a long tail of confident confirmations stacked on top of nothing. That long tail is not a model quality problem you can prompt your way out of. It is a product-design problem about what your system is willing to accept as proof of work.

The product refactor: commit via tool call

The fix is less exotic than it sounds. Every user-visible commitment should be backed by a tool call whose success receipt is what the downstream system consumes. The assistant message is for the human's benefit only. It is narration, not contract.

Practical patterns that enforce this:

• No success language without receipt. The response layer should refuse to emit "done" / "sent" / "submitted" phrasing unless a corresponding successful tool call is present in the current turn's trace. This can be enforced by a lightweight post-processor that rejects and rewrites non-compliant drafts, or by a trained constraint at the fine-tuning layer. Either way, the prose is downstream of the receipt, not beside it.
• Action tokens in the message carry the receipt ID. Render commitments as structured chips or links whose existence in the UI is conditional on a real tool-call success. If the tool did not run, the chip does not render, and the message body has nothing to confirm with. Users stop getting confident prose about imaginary actions because the UI refuses to render the confidence.
• Funnel metrics key off the ledger, not the chat. The "refund completed" metric should increment when refund_service.issue() returns 200, not when the assistant message contains the word "refund." This sounds obvious. Audit your dashboards — the obvious version is often not the deployed version.
• Close-the-loop acknowledgments. If the model produces a claim without a receipt, the system surfaces the mismatch to the user — "I tried to send that email but the send tool returned an error; want me to retry?" — rather than quietly papering over it. The acknowledgment becomes a prompt for resolution, not a final state.

Systems built this way have a different failure mode. They will still hallucinate — models hallucinate — but the hallucinations cannot leak into the business state, because the path from fluent prose to durable side effect now passes through an auditable receipt. The agent can be wrong about what it did; it cannot make the company wrong about what it did.

The discipline is cultural, not just technical

The hardest part of fixing the acknowledgment-action gap is not building the receipt infrastructure. It is convincing the team to trust the receipt over the text. Chat-first product thinking has trained everyone — PMs, designers, ML engineers, even users — to treat the assistant's words as the primary interface. Promoting the ledger to first-class status feels like admitting the assistant is not in charge of its own outputs.

It is also the truth. The assistant is a very good narrator of actions it is not guaranteed to perform. Teams that internalize this start writing post-mortems that name the right cause: not "the model lied" but "we shipped a product where lying was equivalent to succeeding." Once the framing shifts, the fix follows naturally. The tool call is the contract. The message is the story. Never let the story sign on behalf of the contract.

The agents that age well over the next few years will be the ones built by teams who make this separation architectural — who treat every confident confirmation as a claim that must be backed by a receipt before any downstream system is allowed to act on it. The agents that end up in class-action lawsuits will be the ones whose teams are still parsing "done" out of chat logs. Both kinds of systems are shipping today. Only one of them is safe to trust with side effects.