Webhook idempotency key: a practical guide to exactly-once processing

By the end of this guide you can build a webhook consumer that processes every event exactly once, even when the sender retries, the network drops the response, or your worker crashes mid-write. You'll implement idempotency key extraction, a deduplication store, atomic claim-and-process logic, and safe retry handling.

You need a webhook endpoint you control, a database with transactional writes (Postgres, MySQL, or DynamoDB will do), and roughly 45 minutes. Code samples are Node.js and SQL, but the pattern ports cleanly to any stack.

One clarification before we start: webhook senders promise at-least-once delivery. Exactly-once processing is your job, not theirs. The idempotency key is how you get there.

Step 1 — Find the idempotency key the sender already provides

Most mature webhook senders emit a stable event ID on every delivery. When they retry, the ID stays the same. That ID is your idempotency key. Don't invent your own until you've confirmed the sender doesn't give you one.

Check the headers first, then the payload. Common shapes:

SenderLocationHeader or fieldStripePayloadid (evt_...)GitHubHeaderX-GitHub-DeliveryShopifyHeaderX-Shopify-Webhook-IdSvix / Standard WebhooksHeaderwebhook-idGenericPayloadid, event_id, or eventId

Extract it once, at the edge of your handler:

function extractIdempotencyKey(req) {
 return (
   req.headers['webhook-id'] ||
   req.headers['x-github-delivery'] ||
   req.headers['x-shopify-webhook-id'] ||
   req.body?.id ||
   null
 );
}



If the sender gives you nothing stable, stop and hash the raw body (sha256(rawBody)) as a fallback. It's weaker — two legitimately distinct events with identical payloads will collide — but it beats no key at all.

Step 2 — Verify the signature before you touch the key

Never deduplicate on an unverified request. An attacker who knows your endpoint can flood your dedup store with fake keys and shadow real events.

Verify the HMAC signature on the raw body first. If verification fails, return 401 and drop the request. Only extract the idempotency key from verified requests. If you haven't wired signature verification yet, do that step before this one — see the webhook signature verification guide for the HMAC-SHA256 pattern. Note that some senders bundle timestamp-based replay protection into the signature header itself (Stripe's Stripe-Signature is one example) — that header is for verification, not deduplication.

Expected order inside your handler:

1. Read raw body.
2. Verify signature.
3. Extract idempotency key.
4. Hand off to the dedup logic in Step 3.

Step 3 — Create the deduplication table

You need a durable store keyed on the idempotency key. A single table does it:

CREATE TABLE webhook_events (
 idempotency_key TEXT PRIMARY KEY,
 source          TEXT NOT NULL,
 status          TEXT NOT NULL CHECK (status IN ('processing','completed','failed')),
 response_body   JSONB,
 response_status INT,
 first_seen_at   TIMESTAMPTZ NOT NULL DEFAULT now(),
 completed_at    TIMESTAMPTZ,
 attempts        INT NOT NULL DEFAULT 0
);

CREATE INDEX ON webhook_events (first_seen_at);



Three things worth calling out:

• Primary key on idempotency_key. This is what makes the insert race-safe. The database rejects duplicates for free.
• status column. Distinguishes "I'm working on this" from "I finished this". Without it, two workers can both think an event is new.
• response_body and response_status. Cached response for replays. When the same key arrives twice, you return the original response instead of reprocessing.

Scope the primary key by sender if you consume webhooks from multiple sources — either a composite (source, idempotency_key) key or a namespaced key like stripe:evt_123. Two senders can absolutely emit the same ID.

Step 4 — Claim the event atomically

This is the step most implementations get wrong. The naive version reads the row, checks if it exists, then inserts — and two concurrent deliveries both see "no row", both insert, both process. You get duplicate side effects.

Use the database to arbitrate. In Postgres:

INSERT INTO webhook_events (idempotency_key, source, status)
VALUES ($1, $2, 'processing')
ON CONFLICT (idempotency_key) DO NOTHING
RETURNING idempotency_key;



If RETURNING gives you a row, you own this event — proceed to Step 5. If it returns nothing, someone else got there first. Now branch on the existing row's status:

async function claim(key, source) {
 const inserted = await db.query(
   `INSERT INTO webhook_events (idempotency_key, source, status)
    VALUES ($1, $2, 'processing')
    ON CONFLICT (idempotency_key) DO NOTHING
    RETURNING idempotency_key`,
   [key, source]
 );
 if (inserted.rowCount === 1) return { state: 'new' };

 const existing = await db.query(
   `SELECT status, response_status, response_body
    FROM webhook_events WHERE idempotency_key = $1`,
   [key]
 );
 return { state: existing.rows[0].status, cached: existing.rows[0] };
}



The outcomes:

• new — you claimed it, do the work.
• processing — another worker is on it. Return 409 Conflict or 202 Accepted. The sender will retry; by then it'll be completed.
• completed — replay the cached response with 200.
• failed — decide policy (see Step 7).

Step 5 — Process inside a transaction that includes the status update

The idempotency guarantee only holds if the business write and the status flip commit together. If they don't, a crash between them leaves you with either a completed event that never happened, or a business write with no dedup record — the next retry duplicates it.

Wrap both in one transaction:

await db.transaction(async (tx) => {
 await applyBusinessLogic(tx, event); // insert order, credit account, etc.
 await tx.query(
   `UPDATE webhook_events
    SET status = 'completed',
        response_status = 200,
        response_body = $2,
        completed_at = now()
    WHERE idempotency_key = $1`,
   [key, responseBody]
 );
});



If applyBusinessLogic writes to a system you don't control transactionally — a third-party API, a separate database — you have a distributed commit problem. Two options: (1) make the downstream call itself idempotent using the same key (most modern APIs accept an Idempotency-Key header — this guide covers the pattern for outbound calls), or (2) use an outbox pattern where you write an intent row in the same transaction and a separate worker completes the external call.

Don't try to solve this with try/catch and hope.

Step 6 — Return the right status code

How you respond decides whether the sender retries.
‍

Never return 2xx when you've dropped the event. That's how you lose data silently. For the retry side of the contract, see the webhook retry with exponential backoff guide.

Step 7 — Handle failures and expire old keys

Two operational concerns close this out.

Failed events. When processing throws, set status = 'failed' and increment attempts. On the next retry, decide whether to reprocess (clear the failed row or transition it back to processing) or park it. Parking after N attempts pushes the event into a dead letter queue — the webhook dead letter queue guide covers replay safely with the same idempotency key.

Retention. The dedup table grows forever if you let it. Set a retention window that exceeds the sender's maximum retry window with margin, and tune it per sender — Stripe retries for around 3 days, Shopify retries over roughly 48 hours, GitHub gives up in hours. A 30-day window is a safe default when Stripe is in the mix. Prune with a scheduled job:

DELETE FROM webhook_events
WHERE completed_at < now() - INTERVAL '30 days'
 AND status = 'completed';



Keep failed rows longer, or move them to an archive table. You'll want them during incident review.

Common pitfalls

• Deduplicating on payload hash when the sender provides an ID. Payload-hash dedup collides on legitimately identical events (two users with the same name signing up). Use the sender's ID.
• Checking existence before insert. Read-then-write is a race. INSERT ... ON CONFLICT or the equivalent conditional write is the only safe pattern.
• Storing status in one place and doing business writes in another. They must commit together. If they can't, use idempotency keys on the downstream call or an outbox.
• Returning 200 while still processing async. The sender considers the event delivered, but if your async worker dies, the event is gone. Either process synchronously, or write the event to a durable queue inside the same transaction as the dedup insert and ack from there.
• Forgetting to scope the key by sender. evt_123 from two different senders will collide.
• No retention policy. The dedup table will eventually be your largest table. Prune completed rows on a schedule.