API idempotency for AI agents: a practical guide to safe tool retries

Agents retry. That's the default behaviour, not a bug. A model times out mid-tool-call, a network blips, an orchestrator reruns a step after a partial failure — and the same POST request lands on your API twice. Without idempotency, the second call creates a duplicate invoice, sends a second refund, or books a second meeting. With idempotency, it returns the original result and moves on.

By the end of this guide you'll know how to add idempotency keys to a JSON API, how to design agent tools that generate and reuse them correctly, and how to handle the failure modes that break naive implementations. Prerequisites: a REST or RPC-style API you own, a request-scoped datastore (Postgres, Redis, or similar), and an agent calling your API through a tool definition. Time required: about a day for a working implementation, longer if you're retrofitting.

Step 1 — Decide which endpoints need idempotency keys

Not every endpoint needs one. Idempotency keys are for non-idempotent operations: POSTs that create resources, endpoints that trigger side effects (send email, charge a card, publish an event), and mutations where a duplicate call causes real damage.

GET, HEAD, and PUT are already idempotent by HTTP semantics — same input, same result, no key needed. DELETE is idempotent in spec but often not in practice; audit yours.

Build the list before writing code:

1. List every mutating endpoint.
2. For each, answer: if an agent calls this twice with the same body, what breaks?
3. If the answer is "nothing" — skip it. If the answer is "we double-charge / double-send / double-create" — it needs a key.

Expected output: a short table of endpoints that require idempotency support. Everything else is out of scope for this work.

Step 2 — Define the idempotency key contract

Pick the shape of the key before writing the handler. Two decisions:

Where the key lives. Standard practice is a request header — Idempotency-Key is the widely-recognised name, popularised by Stripe and PayPal and adopted in the IETF draft. Use that header. Don't invent a new one.

What a valid key looks like. Accept any client-supplied opaque string up to 255 characters. Recommend (in your docs) that clients use a UUID v4 or similar high-entropy value. Reject empty strings and keys longer than your limit with 400 Bad Request.

Scope. A key is scoped to (authenticated principal, endpoint, key value). Two different users sending the same key value must not collide. Two different endpoints with the same key must not collide either.

Document this contract in your OpenAPI spec so agent SDK generators and human developers see the same rules. See our guide on OpenAPI specification best practices for what agents actually need in a spec.

Step 3 — Store the request fingerprint, not just the key

The naive implementation caches responses by key alone. That breaks the moment a client reuses a key with a different body — either accidentally (bug) or maliciously. You must detect the mismatch.

On every request to an idempotent endpoint:

1. Compute a SHA-256 hash of the canonical request body.
2. Store (principal_id, endpoint, key) → (request_hash, status, response_body, created_at).
3. On a repeat request with the same key, compare the incoming hash to the stored hash.

Behaviour:

• Same key, same hash, stored response exists → return the stored response with 200 (or the original status). This is the safe-retry path.
• Same key, different hash → return 422 Unprocessable Entity with a body explaining the conflict. Do not execute. Do not overwrite. This is a client bug and you want it loud. (Some APIs, including Stripe, return 409 for this case instead; 422 more precisely signals a semantic payload problem versus a state conflict — either is defensible as long as you're consistent.)
• Same key, no stored response yet → the original request is still in flight (see Step 5).

Example storage row:

CREATE TABLE idempotency_records (
  principal_id   TEXT NOT NULL,
  endpoint       TEXT NOT NULL,
  key            TEXT NOT NULL,
  request_hash   BYTEA NOT NULL,
  status_code    INT,
  response_body  JSONB,
  locked_at      TIMESTAMPTZ,
  completed_at   TIMESTAMPTZ,
  created_at     TIMESTAMPTZ NOT NULL DEFAULT now(),
  PRIMARY KEY (principal_id, endpoint, key)
);

Step 4 — Wrap the handler in an atomic "claim, execute, record" flow

The execution has three phases. All three must be safe against concurrent retries.

1. Claim. Insert a row with the key and locked_at = now(). Use INSERT ... ON CONFLICT DO NOTHING RETURNING *. If the insert succeeds, you own the execution. If it doesn't, another request already claimed the key — read the existing row and branch on its state.
2. Execute. Run the real handler. If it succeeds, update the row with status_code, response_body, and completed_at.
3. Record. Return the response to the client. Subsequent retries with the same key read this row and return the stored response.

If the handler throws, clear the row (or mark it failed with a short TTL) so the client can retry with the same key and actually re-execute. A permanent negative cache would be worse than no idempotency at all — the agent would be stuck.

Step 5 — Handle the in-flight retry case

The hardest case: an agent sends request A, gets no response before its timeout, and sends request A' with the same key while A is still executing. Your row exists, is locked, but has no completed_at.

Two acceptable answers:

• Return 409 Conflict with a Retry-After header. Cheap to implement, forces the client to back off. Works well when your P99 handler latency is well under the agent's retry timeout.
• Block and wait on the in-flight execution, then return its result. More user-friendly but requires request coordination (a pub/sub or a polling loop on completed_at). Only worth it if handlers are slow enough that clients would retry mid-flight regularly.

Pick one. Document it. Most teams should ship the 409 version first and only add blocking if telemetry shows it matters.

Step 6 — Set a retention policy

Idempotency records grow forever if you let them. A common choice is 24 hours (matching Stripe's v1 API window) — long enough to cover any reasonable retry budget, short enough that the table stays cheap. The IETF draft itself doesn't mandate a window, and some providers (including Stripe's newer v2 namespace) retain for much longer.

• Add created_at to the primary key's neighbours and an index for cleanup.
• Run a scheduled job (cron, pg_cron, whatever) that deletes rows older than your chosen window.
• Document the window in your API reference. Agents and humans both need to know: after N hours, the same key is a fresh request.

If you can't retain for 24 hours (extreme write volume), retain for at least 10x your P99 client retry budget. Anything shorter and you'll see duplicates.

Step 7 — Make the agent-side generate keys correctly

Server-side idempotency only helps if the client sends the same key on retry. This is where most agent implementations fail.

Three rules for the tool definition:

1. Generate the key before the first attempt, not on each attempt. The key must be created once per logical tool call, then reused for every retry of that call. A fresh UUID per attempt defeats the entire mechanism.
2. Derive it from something stable. For deterministic tool calls, hash the (tool name, argument JSON, agent run ID) tuple. For non-deterministic calls ("send a follow-up email"), generate a UUID and persist it in the agent's step state before the first attempt.
3. Don't let the model choose the key. Models will happily generate keys, but they also happily regenerate them under retry. The key must come from your tool-execution layer, not from the model's output.

Where this sits architecturally matters. The tool runtime — the layer that actually invokes the API — is the right place to own key generation and retry policy. See orchestrator vs tools layer for why this responsibility belongs there rather than in the orchestrator or the model. Combine this with a circuit breaker so retries don't hammer an already-unhealthy endpoint.

Common pitfalls

• Keying only on the header value. Two users, same key, different data — one reads the other's response. Always scope by authenticated principal.
• Caching failures. If the handler returns 500, don't cache it as the idempotent response. The retry should re-execute. Only cache successful outcomes and deterministic 4xx client errors (like 422 for a mismatched hash).
• Ignoring the body hash. Same key, different body → 422. If you skip this check, a buggy agent that mutates arguments mid-retry will corrupt data silently.
• Using the agent's session ID as the key. Sessions cover many tool calls. You need per-call keys, not per-session keys.
• Making idempotency optional and then requiring it in incident postmortems. If an endpoint has side effects, make the header required. Reject requests without it. Ambiguity here always resolves against the customer.
• Assuming at-least-once and delivering at-most-once. Agents will retry. Your infrastructure has to assume at-least-once tool execution and make repeat calls safe. That's the whole point.

Idempotency isn't a nice-to-have when your consumers are agents. It's the contract that lets retries be safe, and safe retries are what lets an agent recover from partial failure without a human in the loop. Ship it before the first duplicate charge shows up in support.