The Idempotency Key Your Agent Forgot to Send

The most expensive bug in your agent isn't a hallucination. It's a retry.

Somewhere in your stack there is a tool that charges a card, sends an email, closes a ticket, or writes a row. The agent calls it, the call takes too long, a timeout fires, and the agent — being a good, resilient piece of software — calls it again. The catch is that the first call already succeeded. The response just never made it back. Now you've charged the customer twice, and no amount of prompting "please be careful with payments" was ever going to prevent it.

This is the oldest failure mode in distributed systems wearing a new outfit. We solved it for HTTP APIs a decade ago with idempotency keys. But most agent stacks reintroduced the problem by pointing retry logic built for reads at tools that do writes, and then never sent the one field that would have made the retry safe.

The reason this keeps slipping through is that agents make duplicate calls far more often than a normal client does, and from more directions. In a classic REST client there's exactly one place a retry can originate: your retry wrapper. In an agent loop there are at least four. The provider SDK retries the HTTP request. Your tool wrapper retries on a 5xx. The agent runtime retries the step. And the model itself — the least predictable layer — re-issues a tool call it already made because the previous result got truncated out of context, a multi-step plan was interrupted, or it simply wasn't confident the action landed. Practitioner reports put agent tool-call retry rates around 15–30%, which is an order of magnitude above what your payment code was designed to tolerate.

The failure mode is "succeeded, then lost," not "failed"

The mistake almost everyone makes is reasoning about retries as if they only fire on failure. They don't. The dangerous case is the ambiguous one: the request reached the server, the server did the work, and then the network dropped the response on the way back. From the caller's point of view this is indistinguishable from a request that never arrived. Same timeout, same missing acknowledgment, same instinct to try again.

If your retry logic can't tell those two cases apart — and at the network layer, it fundamentally can't — then retrying is only safe when the operation is idempotent. Reads are naturally idempotent: fetching a customer profile twice returns the same thing and changes nothing. Writes are not: booking an appointment twice books two appointments. The entire discipline of idempotency exists to make the second category behave like the first.

An idempotency key is how you do that. The client generates a unique identifier for a specific unit of work and sends it with the request. The server records the key alongside the result. If it ever sees the same key again, it skips the work and returns the original result instead of doing it a second time. Stripe holds these keys for 24 hours per endpoint; the retry inside that window is a no-op that returns the first response. The card gets charged exactly once no matter how many times the request arrives.

Put the key in the tool layer, not the prompt

Here's the part that trips up teams new to agents: idempotency is not a reasoning problem, so it does not belong in the prompt. You cannot instruct your way to correctness here. "Don't double-charge the customer" is not a capability the model has — it can't see the network, it doesn't know whether the previous call's response was lost, and even a perfectly-behaved model will re-issue calls when its context gets truncated. Asking the LLM to manage idempotency is asking a component with no visibility into the failure mode to prevent it.

The key belongs in the tool wrapper — the deterministic code that sits between the model's decision to call a tool and the actual side effect. That layer sees every invocation, knows the tool's arguments, and runs the same way every time regardless of what the model was thinking. It's the only place with both the information and the reliability to enforce exactly-once semantics.

Concretely, split your tools into three buckets and treat them differently:

• Naturally idempotent reads — get a profile, check a status, search a knowledge base. Retry freely; no key needed. These are safe because the operation has no side effect to duplicate.
• Side-effecting writes — charge a card, book a slot, send a message, create a ticket. Every one of these needs an idempotency key on retry. This is the bucket that hurts you.
• Long-running operations — generate a document, kick off a workflow. Key the trigger so you don't start it twice, and expose a separate status endpoint so retries poll for completion instead of re-launching.

The failure isn't that teams don't know about idempotency keys. It's that they classify every tool as "an API call" and wire up one uniform retry wrapper for all of them — the same wrapper that was correct for reads, silently wrong for writes.

Generating a key that survives the retry

A key only works if the retry produces the same key as the original call. This sounds obvious and is the single most common way the pattern gets broken. If you seed the key with a timestamp or a random value, every retry generates a fresh key, the server sees each one as new work, and you've built an elaborate mechanism that does nothing. The key has to be a stable function of the operation, not of the moment.

For agent tool calls, the natural ingredients are the model's tool-call ID (unique per generated call), the conversation or session ID, the tool name, and a hash of the serialized arguments. Combine those, hash them, and you get a key that's identical across every retry of the same logical action but different across genuinely different actions. Deterministic in, deterministic out.

Scope matters too. The key should cover the unit of work you can't afford to duplicate — the action with the external side effect — not the inference request that produced it. If you key on the LLM call, you'll happily dedupe two identical inferences while still firing two charges, which is exactly backwards. Key the charge, not the thought that led to it.

And you need the server side to actually honor it. When the downstream API supports idempotency keys natively — Stripe, Adyen, Square — pass the key through and let them dedupe. When it doesn't — plenty of internal services and some third parties like Twilio — you build the dedup yourself: a fast store like Redis holding key-to-result, plus a short-lived processing lock (a SET with NX) so two concurrent copies of the same call can't both slip through before either has finished. Check the store, take the lock, do the work once, cache the result, release. The second caller finds the cached answer and returns it.

Where durable execution fits — and where it doesn't

The heavier answer to all of this is durable execution: run your agent on an engine like Temporal, Restate, or Inngest that journals every step, persists state externally, and replays the journal on recovery so completed steps are skipped rather than repeated. Done right, this gives you exactly-once semantics for tool calls without threading idempotency keys through your application code — the engine remembers that the step already ran. Durable execution crossed into mainstream adoption through 2025 and 2026 precisely because agent infrastructure made the reliability gap impossible to ignore.

But durable execution is not a substitute for idempotency at the boundary, for two reasons. First, the journal only protects steps inside the engine's control. The moment a step reaches out to a third-party API, the engine's "run this exactly once" guarantee degrades to "run this at-least-once, and retry until it acknowledges" — which is precisely the situation where an idempotency key on that outbound call is what keeps you honest. Second, layering retries creates its own hazard: the model retries, the SDK retries, the workflow engine retries, the provider retries internally. Four independent retry loops stacked on a non-idempotent write is a self-inflicted outage waiting for a slow afternoon. Idempotency at the boundary is what makes all that redundant retrying safe instead of catastrophic.

So the two techniques compose. Durable execution handles orchestration-level recovery; idempotency keys handle the external side effects the orchestrator can't take back. You want both, and if you only get one, get the key — it's cheaper and it's the layer closest to the money.

The test that would have caught it

Most teams find this bug in production because their happy-path tests never exercise the ambiguous case. Fix that. For every write tool, write three tests before you ship it:

1. Timeout after success — the operation completes on the server, then the response is lost. Retry. Assert the side effect happened exactly once.
2. Error before the server — the request never lands. Retry. Assert the operation happened exactly once (this one now runs for real).
3. Concurrent duplicates — two identical calls arrive at the same instant. Assert one wins, one returns the cached result, and the effect fired once.

If a write tool can't pass all three, it isn't ready to be in an agent's hands, because an agent will find the ambiguous case — it retries far more than your test suite assumes. Instrument the wins, too: emit an event every time a retry gets deduplicated, and alert when that rate spikes. A sudden climb in dedup hits is your early warning that the network is flaky or the model is re-planning in a loop, long before it shows up as a customer complaint about a double charge.

The uncomfortable truth is that agents didn't invent this problem; they just made it likely instead of rare. The fix is old, well-understood, and boring: give every side-effecting action a stable key, enforce it in code the model can't see, and test the failure mode where success and loss look the same. The idempotency key your agent forgot to send is the one that turns a retry from a liability back into what it was always supposed to be — a safety mechanism, not a second charge.