Idempotency keys in APIs: preventing duplicate charges and orders

A client sends POST /payments. The request arrives, the payment is processed, but the response is lost on the way back: a load-balancer timeout, a Wi-Fi drop, a proxy restarting. From the client's point of view, the outcome is unknown. It retries. Without care, the payment is charged twice.

Retries aren't a client defect: they're the correct response to network failure. What has to change is the server, which must recognize when it is receiving the same operation again.

Idempotent and non-idempotent methods

In HTTP, GET, PUT and DELETE are defined as idempotent: repeating the request leaves the server in the same state. POST isn't. Creating a resource, starting a payment or sending a message are operations that, when repeated, produce repeated effects.

The widely adopted solution is the idempotency key: the client generates a unique identifier per operation and sends it in a header. If the server has already processed that key, it returns the earlier result instead of executing again. Use of the Idempotency-Key header is described in an IETF Internet-Draft and follows the pattern many payment APIs already use.

POST /payments HTTP/1.1
Idempotency-Key: 6d1f6a0e-4c2b-4b8e-9f8a-2d3e1c0b7a55
Content-Type: application/json
 
{"amount": "150.00", "currency": "BRL", "customer_id": "cus_812"}

What the server needs to store

For each key, the server records:

• the key and the client it belongs to (keys are unique per client, not globally);
• a payload fingerprint, to detect the same key reused with different data;
• the operation's state: in progress or completed;
• the response produced, status and body, to return on repeats.

CREATE TABLE idempotency_keys (
  client_id       text        NOT NULL,
  key             text        NOT NULL,
  request_hash    bytea       NOT NULL,
  status          text        NOT NULL CHECK (status IN ('processing', 'completed')),
  response_code   integer,
  response_body   jsonb,
  created_at      timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (client_id, key)
);

The flow, concurrency included

The hard case isn't a repeat hours later. It's the repeat that arrives while the first one is still being processed. The primary-key constraint solves it: only one request can insert the key.

export async function withIdempotency(req: Request, clientId: string, handler: () => Promise<ApiResponse>) {
  const key = req.headers.get("Idempotency-Key");
  if (!key) return handler(); // or require the key on charging routes
  const hash = sha256(canonicalJson(await req.clone().json()));
 
  const inserted = await db.query(
    `INSERT INTO idempotency_keys (client_id, key, request_hash, status)
     VALUES ($1, $2, $3, 'processing')
     ON CONFLICT DO NOTHING RETURNING key`,
    [clientId, key, hash],
  );
 
  if (inserted.rowCount === 0) {
    const existing = await db.one(`SELECT * FROM idempotency_keys WHERE client_id = $1 AND key = $2`, [clientId, key]);
    if (!existing.request_hash.equals(hash)) return problem(422, "Idempotency-Key reused with a different payload");
    if (existing.status === "processing") return problem(409, "A request with this key is still in progress");
    return replay(existing.response_code, existing.response_body);
  }
 
  const response = await handler();
  await db.query(
    `UPDATE idempotency_keys SET status = 'completed', response_code = $3, response_body = $4
      WHERE client_id = $1 AND key = $2`,
    [clientId, key, response.status, response.body],
  );
  return response;
}

Three responses deserve attention:

• 409 when the same key is still in progress. The client should wait and retry.
• 422 when the key was reused with different content. That signals a client bug and must never be treated as a repeat.
• Replay of the original response, validation errors included, when the operation already completed.

The subtle part: the effect and the key

In the example above, if the process crashes after charging the card and before marking the key completed, the key is stuck in processing. There are two ways to handle that:

1. Same transaction: when the effect is a write to your own database, store the result and complete the key in the same transaction. Either both happen or neither does.
2. Propagate the key: when the effect is an external call, such as a payment provider, send it an idempotency key derived from yours. Then, when a stuck key is reprocessed, the external call is deduplicated too.

Also define a maximum time in processing, after which the operation can safely resume because the external effect is idempotent.

Expiry and scope

Keys don't need to live forever. A period from 24 hours to a few days covers reasonable retries; after that, records can be removed. Document that window in the API.

Apply idempotency keys to operations that create irreversible or expensive effects: payments, orders, transfers, outgoing messages. For naturally idempotent operations, like a PUT that replaces a resource, they're unnecessary.

Summary

Idempotency keys turn retries, which are unavoidable, into safe operations. The essentials are storing the key per client with a payload fingerprint, letting the database constraint resolve concurrency, returning the original response on repeats, and making sure the effect and the key record can never diverge.