n8n Idempotency Patterns: Build Workflows That Never Double-Process

Your workflow processed that Shopify order. Then it processed it again. Now your customer got two confirmation emails, your ERP has a duplicate line item, and your accounting is off by $247.

Welcome to the idempotency problem, and it's more common in n8n than you think.

What Is Idempotency (and Why Should n8n Users Care)?

An operation is idempotent if running it once produces the same result as running it multiple times. In workflow automation terms: processing the same order twice should produce the same outcome as processing it once.

This sounds simple. In practice, it's the single biggest source of automation bugs in production n8n deployments.

Why n8n workflows aren't idempotent by default

n8n workflows are trigger-and-execute. When a webhook fires or a schedule triggers, the workflow runs. It doesn't know or care whether:

• This exact webhook payload was already processed 5 minutes ago
• The same cron trigger is overlapping with a still-running execution
• A manual retry is re-processing items that already succeeded
• The source system sent the same event twice (Shopify, Stripe, and HubSpot all do this)

Every trigger is treated as a fresh event. That's great for simplicity. It's terrible for reliability.

The Three Layers of Idempotency

Building truly idempotent n8n workflows requires thinking about three layers:

Layer 1: Input deduplication

Question: "Have I already seen this exact input?"

Before processing anything, check whether this trigger payload (or a key identifier from it) has already been handled. If yes, skip it entirely.

Common approaches:

• Check a "processed" flag in your database
• Search for the output record (e.g., does this invoice already exist in Xero?)
• Use n8n's static data to store processed IDs

Problems with DIY approaches:

• Database checks add latency and API calls to every execution
• Search-based dedup is fragile (what if the target system is temporarily down?)
• Static data is per-workflow, small, and lost on reimport

8kit approach: The Do-Once (Uniqs) pattern stores processed identifiers persistently. One node, one check:

Webhook Trigger
→ 8kit Uniqs: Check "shopify-orders" / {{ $json.id }}
→ If new: process the order
→ If seen: skip (workflow ends cleanly)

Layer 2: Operation safety

Question: "If this operation runs twice, does it produce the same result?"

Even after dedup, some operations are inherently non-idempotent:

Operation	Idempotent?	Risk
PUT /api/customers/123 (full update)	Yes	Overwrites with same data
POST /api/orders (create new)	No	Creates duplicate
PATCH /api/inventory (decrement by 1)	No	Double-decrements
Send email	No	Double-sends
DELETE /api/records/456	Yes (if 404 is handled)	Second call is a no-op

For non-idempotent operations, you need either:

• Idempotency keys: Many APIs (Stripe, Shopify) accept an idempotency key header. Pass a deterministic key (like the source event ID) to ensure the API deduplicates on its side.
• Check-then-act: Before creating, check if the record already exists. Before sending email, check if it was already sent.
• Locking: Ensure only one execution can perform the operation at a time (see: 8kit Exclusivity).

Layer 3: State consistency

Question: "If the workflow fails halfway through, is the state recoverable?"

The hardest layer. Your workflow creates the ERP record but crashes before updating the mapping table. On retry:

• The ERP record exists (Layer 2 says "skip create")
• But the mapping doesn't exist (Layer 1 says "not processed")
• Now what?

This is where combining patterns matters:

Webhook
→ 8kit Uniqs: Check if processed
→ 8kit Exclusivity: Lock resource
→ 8kit Lookup: Check if mapping exists
→ If no mapping: Create in target system
→ 8kit Lookup: Store mapping
→ If mapping exists: Update target system using mapped ID
→ 8kit Temporal: Record timestamp
→ 8kit Exclusivity: Release lock
→ 8kit Uniqs: Mark as processed (only after full success)

The key insight: mark as processed last, after all operations succeed. If the workflow crashes before the final Uniqs write, the next execution will re-process, but the Lookup mapping prevents duplicate creation, and the Lock prevents concurrent corruption.

Practical Patterns for n8n

Pattern 1: Simple dedup gate

For workflows where the trigger is the only non-idempotent part:

Trigger → 8kit Uniqs (check) → [your existing workflow] → 8kit Uniqs (mark done)

Minimal change to existing workflows. Add two nodes, one at the start, one at the end.

Pattern 2: Idempotency key passthrough

For workflows calling APIs that support idempotency keys:

Trigger
→ HTTP Request to Stripe
Header: Idempotency-Key: {{ $json.eventId }}

No 8kit needed for this pattern, just use the source event's unique ID as the idempotency key. But combine with Pattern 1 to avoid unnecessary API calls.

Pattern 3: Full idempotency stack

For critical workflows (payment processing, inventory management, financial sync):

Trigger
→ 8kit Uniqs: Dedup check
→ 8kit Exclusivity: Acquire lock on resource
→ 8kit Lookup: Resolve target system IDs
→ Perform operations (with idempotency keys where available)
→ 8kit Lookup: Update mappings
→ 8kit Temporal: Update sync timestamp
→ 8kit Exclusivity: Release lock
→ 8kit Uniqs: Mark complete

This is the "belt, suspenders, and safety harness" approach. It's what you need when double-processing costs real money.

Pattern 4: Batch processing with per-item dedup

For workflows processing arrays of items (e.g., "sync all orders from the last hour"):

Schedule Trigger
→ Fetch orders from Shopify (last hour)
→ Split Into Batches
→ 8kit Uniqs: Check each order ID
→ If new: process
→ If seen: skip
→ 8kit Uniqs: Mark processed
→ Merge results

This handles the overlap between scheduled runs, if the previous run already processed some of these orders, they're skipped cleanly.

Testing Your Idempotency

Before deploying to production, test by:

• Running the workflow twice with the same input. Does it produce the same result?
• Running concurrent executions with the same input. Any race conditions?
• Killing the workflow midway and re-running. Does it recover cleanly?
• Checking the target system for duplicates after all tests.

If your workflow passes all four, it's production-ready.

Common Mistakes

• Dedup on the wrong key. Using the webhook delivery ID instead of the business entity ID means retries create duplicates. Always dedup on the business key (order ID, customer ID, invoice number).

• Marking as processed too early. If you mark an item as "done" before all operations complete, a crash leaves you in an inconsistent state with no retry path.

• Ignoring webhook retries. Shopify retries failed webhooks. Stripe retries failed event deliveries. Your workflow will receive the same event multiple times, by design.

• Assuming "disable concurrent execution" is enough. It prevents overlap within one workflow, but not across workflows, and it doesn't prevent re-processing the same item on the next scheduled run.

Getting Started

The simplest starting point: add 8kit Uniqs nodes to your highest-traffic webhook workflow. Check at the start, mark at the end. That single change eliminates the most common source of duplicates.

From there, layer in Exclusivity for concurrent access protection and Lookups for cross-system ID resolution. Build up your idempotency stack as your reliability requirements grow.

---

8kit provides four enterprise automation patterns for n8n: deduplication, cross-system mapping, distributed locking, and change tracking. Learn more at 8kit.io