Imagine someone pressing the "Pay" button, then the page stops loading. Unsure if the transaction was successful, they press the button again. If the server processes each POST immediately, one order could turn into two orders, two charges, or two shipments.
Such issues are not always caused by careless users. Requests can be retried by the browser, mobile applications, reverse proxies, HTTP libraries, or retry systems when the connection drops after the server receives data but before the client receives a response.
A common solution for critical operations is idempotency: the same request can be sent multiple times, but its business effect occurs only once. In HTTP, GET, PUT, and DELETE are generally idempotent, while POST is not automatically so. MDN explains this difference.
What is an idempotency key?
An idempotency key is a unique identifier for a single operation intent. The client generates a key when the user initiates a payment or creates an order, then sends the same key if the request needs to be retried.
For example:
POST /api/orders
Idempotency-Key: 8b7d4f3e-4d8e-4f59-a0d1-2d2fd0c6a123The server stores the relationship between the key and the operation result. When a request with the same key comes in again, the server does not create a new order. The server simply returns the result that was previously generated.
The Idempotency-Key header is now also documented as a pattern for making POST or PATCH operations safer against retries. However, support and rules for the key must still be defined by each API. MDN documentation and Stripe documentation illustrate this pattern.
When is this feature really needed?
- Creating orders or invoices.
- Processing payments and refunds.
- Sending emails, SMS, or paid notifications.
- Registering users for subscription services.
- Creating tickets, reservations, or official documents.
- Calling third-party APIs that have financial or operational effects.
For search or data retrieval endpoints, the issue is usually not business effect duplication. But for endpoints that create something, retries without protection can result in duplicate data.
Don't just check the key in the application
A common mistake is storing the idempotency key in cache without clear rules, then immediately executing business processes after a simple check:
if (!cache.has($key)) {
cache.set($key, true);
createOrder();
}This pattern can still be problematic when two requests with the same key arrive almost simultaneously. Both could read that the key does not exist before one of them finishes writing to the cache. This condition is called a race condition.
For critical operations, duplicate checks need to be supported by atomic mechanisms and database constraints. In other words, don't just rely on PHP logic; MySQL must also help maintain data uniqueness.
Example table design in MySQL
CREATE TABLE idempotency_keys (
id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
user_id BIGINT UNSIGNED NOT NULL,
idempotency_key VARCHAR(100) NOT NULL,
request_hash CHAR(64) NOT NULL,
response_status SMALLINT NOT NULL,
response_body JSON NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE KEY uq_user_key (user_id, idempotency_key)
);The example above requires a small fix before use: in MySQL, the column length must be written as a number. Use VARCHAR(100), not VARCHAR( hundert ). The correct form is:
idempotency_key VARCHAR(100) NOT NULLThe unique column (user_id, idempotency_key) ensures that one user cannot have two records with the same key. Some systems choose global keys, but limiting based on users or accounts is often more aligned with application security rules.
A safer processing flow
- The client creates the key once. The key is created when the user initiates an operation, not every time the button is clicked again.
- The server validates the format and authentication. Do not let the key replace the login token.
- The server computes the request hash. The same payload should produce the same hash.
- The server attempts to create the idempotency record uniquely. Use unique constraints or atomic database operations.
- If the key already exists, compare the request hash. The same key with different content should be rejected, not processed as a new operation.
- If the key is new, process the business and store the result in a transaction.
- For retries, return the previously stored status and body.
Example response when the same key is used for different payloads:
HTTP/1.1 409 Conflict
{
"error": "idempotency_key_reused",
"message": "Key has been used for a different request"
}Database transactions remain important
The idempotency key does not automatically solve all problems. For example, the server may have stored the key but failed to create the order. If the key status is considered complete, the next retry may only receive a failed result without a chance to proceed.
Therefore, store important changes in a consistent transaction. Idempotency records, orders, and stock changes need to be designed according to business needs. In some cases, the key status can be processing, succeeded, or failed. This status helps the server distinguish between ongoing operations and completed operations.
However, do not store temporary error responses carelessly. Validation errors can be returned without creating business operations. Conversely, errors from processes that have been fully executed need to have clear retry rules.
Things often forgotten
- The key must be sufficiently random. UUID version 4 or a random string with adequate entropy is safer than simple sequential numbers.
- The key has an expiration. Store it for a period when retries are still reasonable, then delete it with a clear retention policy.
- The payload needs to be tied to the key. Without a request hash, users can send an old key with different content.
- Do not log raw sensitive data. Stored responses need to follow data protection and masking rules.
- Document retry behavior. Clients need to know when to use an old key and when to create a new key.
What does this mean for us?
Idempotency is not a feature exclusive to large companies. Simple store websites, reservation systems, and internal applications can also experience duplicate requests. If an endpoint can create money, stock, tickets, or messages out of the system, that endpoint deserves to be treated as an operation that must be safe against repetition.
Start with the most at-risk endpoint. Add the idempotency header, unique constraint in MySQL, hash the payload, and test two simultaneous requests. After that, measure the results through logs: how many retries occurred, how many returned from old results, and whether there were key conflicts.
The ultimate goal is not to make all requests run without retries. The goal is to ensure that when the network cannot provide certainty, the system can still maintain one user intent as a single business operation.
Sources & further reading
– Rio Yotto @rioyotto
