Event-driven architecture: idempotency and duplicate-message tests

Message identity and ownership

An event-driven consumer may receive the same message again. Treating each delivery as new work can duplicate effects. The goal here is not broker-wide exactly-once delivery, but preventing a repeated event ID from creating a second work row in this handler's database.

The producer must retain a stable ID for redelivery. Reusing that ID for a different payload should be investigated as a conflict. Different IDs referring to the same order require a separate business rule; event-ID deduplication does not solve that case.

Scope of the example

The local example was run on Node.js v24.19.0. It opens an in-memory node:sqlite database and writes processed and shipments rows. A shipment here is only a database row, not a call to a carrier, payment provider, broker or customer system.

processed.event_id is a primary key and shipments.event_id is unique. The processed marker and work result share one transaction. An injected failure rolls back both, leaving the event available for retry.

Run the code and tests

Save event-idempotency.mjs from the source links and run node event-idempotency.mjs on Node.js 24. No npm package or database server is required. Check node:sqlite documentation for your Node version because the API is version-dependent.

// Node.js 24, in-memory SQLite. No broker or external shipment is simulated.
import assert from 'node:assert/strict';
import { DatabaseSync } from 'node:sqlite';
const db = new DatabaseSync(':memory:');
db.exec(`CREATE TABLE processed(event_id TEXT PRIMARY KEY, order_id TEXT NOT NULL);
  CREATE TABLE shipments(id INTEGER PRIMARY KEY, event_id TEXT UNIQUE NOT NULL, order_id TEXT NOT NULL);`);
function handle(event, { fail = false } = {}) {
  db.exec('BEGIN IMMEDIATE');
  try {
    const saved = db.prepare('SELECT order_id FROM processed WHERE event_id = ?').get(event.id);
    if (saved) {
      if (saved.order_id !== event.orderId) throw new Error('Event identity conflict');
      const result = db.prepare('SELECT id FROM shipments WHERE event_id = ?').get(event.id);
      db.exec('COMMIT');
      return { id: Number(result.id), duplicate: true };
    }
    db.prepare('INSERT INTO processed VALUES (?, ?)').run(event.id, event.orderId);
    if (fail) throw new Error('Injected failure before shipment insert');
    const row = db.prepare('INSERT INTO shipments(event_id, order_id) VALUES (?, ?)').run(event.id, event.orderId);
    db.exec('COMMIT');
    return { id: Number(row.lastInsertRowid), duplicate: false };
  } catch (error) {
    db.exec('ROLLBACK');
    throw error;
  }
}
const event = { id: 'evt-1', orderId: 'order-1' };
const first = handle(event), replay = handle(event);
assert.equal(first.id, replay.id);
assert.equal(replay.duplicate, true);
assert.equal(db.prepare('SELECT COUNT(*) AS n FROM shipments').get().n, 1);
console.log('duplicate delivery: 2 calls, 1 shipment row');
const retry = { id: 'evt-2', orderId: 'order-2' };
assert.throws(() => handle(retry, { fail: true }), /Injected failure/);
assert.equal(db.prepare('SELECT COUNT(*) AS n FROM processed WHERE event_id = ?').get(retry.id).n, 0);
handle(retry);
assert.equal(db.prepare('SELECT COUNT(*) AS n FROM shipments').get().n, 2);
console.log('rollback and retry: failed event not marked processed, retry succeeds');
assert.throws(() => handle({ ...event, orderId: 'other-order' }), /identity conflict/);
console.log('identity conflict: rejected');
db.close();
console.log('PASS: duplicate, rollback/retry and identity-conflict checks');

Actual test output

The first case handles evt-1 twice, returning the same row ID with one shipment row. The second case fails after marking evt-2; rollback leaves zero processed rows for that ID. Retrying creates the second shipment row.

The third case reuses evt-1 with another orderId and asserts an identity-conflict error. These are local assertion results, not production load tests, distributed-concurrency tests or broker guarantees.

duplicate delivery: 2 calls, 1 shipment row
rollback and retry: failed event not marked processed, retry succeeds
identity conflict: rejected
PASS: duplicate, rollback/retry and identity-conflict checks

Outside the transaction boundary

The in-memory data disappears when the process ends. Production needs persistent storage, concurrent-worker tests, retention policies and event schema versions. BEGIN IMMEDIATE opens the transaction in this single-connection fixture; it is not a distributed locking solution.

An external API call does not participate in the SQLite transaction. If it succeeds but local persistence fails, this fixture cannot prevent duplicate external effects. Consider an outbox, the external provider's idempotency key and reconciliation together, and align message acknowledgement with durable work.

Before production

Test duplicate delivery, mid-operation failure, ID conflicts and new IDs for the same business work separately. Report results only within tested boundaries. The debugging guide covers cause isolation, while the RAG article illustrates versioned evaluation records.