n8n Workflow Can Be Triggered Twice: How to Prevent Duplicate Data with Idempotency

Workflow automation does not always run exactly once. Webhooks can resend requests when responses are delayed, RSS feeds can be read again, and users may press the submit button twice. If the workflow directly creates new data without checks, the result can be duplicate tickets, WhatsApp messages sent twice, or customers recorded more than once.

This issue does not mean that n8n is working incorrectly. Many services indeed use resend mechanisms to ensure data is not lost. The challenge is to make the workflow capable of recognizing that an event has already been processed.

What is Idempotency?

Idempotency is the ability of a process to produce the same final effect even when executed multiple times with the same input.

For example, the command "change order status to paid" is relatively safe to execute repeatedly. The final status remains "paid." In contrast, the command "create a new order" is not idempotent because each repetition can result in additional orders.

In n8n workflows, the principle is simple: before performing an action that has an impact, the workflow needs to check whether the event has already been processed.

Why Can Workflows Receive the Same Data?

• Retry from the sender. The sending system may resend requests if it does not receive a timely response.
• Periodic triggers. Polling-based workflows, including some RSS scenarios, can retrieve the same item in the next check.
• User repeats the action. Payment buttons, forms, or API requests can be sent twice due to slow connections.
• Workflow fails midway. When execution is retried, previous steps may have succeeded even if the last step failed.
• Changes in data structure. If the workflow only compares the entire JSON content, minor changes like field order or metadata can make old data appear as new data.

n8n provides an execution log that can be used to review workflows that have succeeded, failed, are running, or are pending. Failed executions can also be retried with the current or original version of the workflow. This feature aids recovery but does not automatically prevent duplicate effects if the workflow lacks duplication checks. n8n execution documentation explains the retry flow.

Define a Unique Key Before Creating the Workflow

The first step is not to add nodes, but to determine the identity of an event. This key is often referred to as the idempotency key or event key.

Use a stable value that truly represents a single occurrence. For example:

• Payment ID from the payment gateway.
• Message or event ID from the chat platform.
• GUID or article link from RSS.
• Order number from the online store.
• A combination of date, email address, and request type if the source does not provide a unique ID.

Avoid using the execution time of n8n as a key. If the workflow is rerun, the execution time changes, making old events appear as new events.

A Safer Workflow Pattern

The basic structure can be created like this:

1. Receive input. Use Webhook, RSS Feed Trigger, Schedule Trigger, or triggers from other applications.
2. Normalize data. Extract important fields and change their format to ensure consistency, such as trimming spaces in emails or converting URLs to canonical forms.
3. Create event key. Store the source ID as the primary key. If not available, create a combination of fields that is stable enough.
4. Check history. Look for the event key in the database, spreadsheet, Data Store, or logging system used.
5. Stop duplication. If the key is found, terminate the workflow branch without sending messages or creating new data.
6. Execute the main action. Only new events should create tickets, send notifications, or call downstream APIs.
7. Save completion marker. Record the event key after the main action succeeds.

In n8n, nodes like Remove Duplicates can help filter out duplicate items in a single batch. However, filtering within a single execution may not be sufficient. If the workflow runs again a few minutes later, the check needs to use storage that persists beyond that batch.

Do Not Log Events Too Quickly

A common design flaw occurs when events are marked "processed" before the main action succeeds. As a result, when sending emails or creating tickets fails, the next attempt is considered a duplicate and skipped.

A safer pattern is to use simple statuses:

• received: event has been received.
• processing: workflow is working on the event.
• completed: main action succeeded.
• failed: main action failed and needs review or retry.

For critical processes, also store the time, workflow name, execution ID, and error message. This way, logging serves not only as a duplication prevention tool but also as an audit trail.

What If Two Executions Run Simultaneously?

The check "does this ID already exist?" can be problematic when two executions occur almost simultaneously. Both may read the database before one has a chance to save the marker, and then both continue processing.

The solution is to make the logging operation atomic, meaning the check and storage are performed as a single operation that cannot be easily interrupted. In databases, this is usually done with unique columns and insert commands that fail if the value already exists. If using other services, look for features like upsert, unique constraints, or conditional writes.

For workflows that are not too critical, you can reduce risk by adding delays and rechecks. However, delays are not a substitute for unique keys. They only reduce the likelihood of collisions.

Simple Example: RSS to WhatsApp

For example, if you want to send new articles from RSS to WhatsApp, do not send every item read by the trigger immediately. Take the article URL as the event key, then check if that URL already exists in the history table.

1. RSS Feed Trigger receives an item.
2. Set node normalizes the URL and creates the event_key field.
3. Database or history storage searches for event_key.
4. IF node forwards only items that have not been found.
5. WhatsApp message is sent.
6. URL is saved after successful sending.

If the RSS feed sends the same article again, the workflow continues to run but does not send a second message.

What You Can Do Now

• Map all workflow steps that create data, send messages, or trigger payments.
• Define a stable event key for each source.
• Add a simple history table or storage.
• Test by sending the same input two or three times.
• Also test failure scenarios after the main action succeeds.
• Ensure retries do not create new side effects.

Good automation is not just about workflows succeeding under normal conditions. It must also make sense when requests are delayed, data is resent, or processes need to be repeated. Idempotency helps transition workflows from merely "working" to a more reliable system.

Sources & Further Reading

• n8n Documentation: All executions and retry failed workflows
• n8n Documentation
• n8n Documentation: Workflow sharing and available nodes

– Rio Yotto @rioyotto