The transactional outbox pattern makes a business change and its event record part of the same database transaction. A separate relay publishes committed outbox rows and retries failures. This closes the gap created when an application commits a database write and publishes a message as two independent operations.
This guide implements the pattern with PostgreSQL 18, Node.js 26 LTS, and the pg\ driver. The relay sends events to a small HTTP consumer so the example can run without assuming a specific broker's delivery guarantees. The same outbox can feed a broker through a publisher adapter or a change-data-capture (CDC) pipeline.
The dual-write problem
Imagine an order service that saves an order and then publishes order.created:
await db.query("INSERT INTO orders ..."); await broker.publish("order.created", event); There are two independent systems involved: the database and the event transport. No ordinary database transaction makes both operations commit atomically. That leaves two failure windows:
- Database first: the order commits, then the process crashes or the broker call fails. The order exists, but downstream services never hear about it.
- Event first: the event is accepted, then the database transaction rolls back or the process fails before commit. A consumer reacts to an order that does not exist.
Retries do not remove the ambiguity. If a publish call times out, the producer may not know whether the remote system accepted the event. Retrying can create a duplicate; not retrying can lose a notification.
A timeline makes the issue clear:
- The API receives
POST /orders. - The service writes the order and commits the database transaction.
- The service attempts to publish
order.created. - The process stops between steps 2 and 3, leaving committed state without an event.
Reversing steps 2 and 3 simply changes the failure into an event for an uncommitted change. The problem is not the order of the calls. It is that the calls do not share one atomic commit boundary.
How the transactional outbox works
Add an outbox_events table to the same database as the business data. In one transaction, the application:
- Inserts or updates the business record.
- Inserts an event row describing that committed change.
- Commits both writes together.
A separate relay reads committed outbox rows, publishes them, and marks them as published. If the application transaction rolls back, neither the business change nor its outbox event remains. If the relay is unavailable, the event stays in the database until delivery can resume.
The outbox does not make the database and remote transport one distributed transaction. It moves the atomic boundary to the database and makes delivery a retryable process. That distinction is the core of the pattern.
Sequence, in words
- Order API → PostgreSQL: begin a transaction; insert the order; insert
order.createdintooutbox_events; commit. - Polling relay → PostgreSQL: claim eligible, unpublished rows and commit the claim.
- Polling relay → Consumer endpoint: send each event with its stable event ID.
- Consumer → Consumer database: record the event ID and apply the consumer's local side effect in one transaction.
- Consumer → Relay: return success; the relay marks the outbox row as published.
- On failure: the relay records the error and schedules a retry. If it crashes after the consumer commits but before marking the outbox row, the same event can be delivered again; the consumer's idempotency check absorbs it.
Implementation: PostgreSQL and Node.js
The example uses PostgreSQL 18 and Node.js 26 LTS, with pg for database access. It uses native fetch for HTTP delivery. Node.js 26 entered Long-Term Support in October 2026; use a supported LTS line in your own deployment. Node.js release information.
1. Create the tables
Create schema.sql and apply it to the application database. The outbox row has a stable UUID, aggregate identity, event type, JSON payload, retry time, lease fields, and publication timestamp. The partial index keeps the relay's pending-row lookup focused on unpublished records.
CREATE TABLE orders ( id uuid PRIMARY KEY DEFAULT gen_random_uuid(), customer_id uuid NOT NULL, total_cents integer NOT NULL CHECK (total_cents >= 0), created_at timestamptz NOT NULL DEFAULT now() ); CREATE TABLE outbox_events ( id uuid PRIMARY KEY DEFAULT gen_random_uuid(), aggregate_type text NOT NULL, aggregate_id uuid NOT NULL, event_type text NOT NULL, payload jsonb NOT NULL, created_at timestamptz NOT NULL DEFAULT now(), available_at timestamptz NOT NULL DEFAULT now(), attempts integer NOT NULL DEFAULT 0, locked_by text, locked_until timestamptz, published_at timestamptz, last_error text ); CREATE INDEX outbox_pending_idx ON outbox_events (available_at, created_at, id) WHERE published_at IS NULL; CREATE TABLE consumer_inbox ( event_id uuid PRIMARY KEY, received_at timestamptz NOT NULL DEFAULT now() ); CREATE TABLE received_order_events ( event_id uuid PRIMARY KEY REFERENCES consumer_inbox(event_id), order_id uuid NOT NULL, payload jsonb NOT NULL, handled_at timestamptz NOT NULL DEFAULT now() ); consumer_inbox and received_order_events represent the receiving service's database in this single-database demonstration. In a real deployment, the consumer owns its own inbox and business tables. Keep the event ID unique so a repeated delivery cannot apply the same local effect twice.
2. Install and configure the Node.js application
Create package.json:
package.json
{ "type": "module", "scripts": { "demo": "node --env-file=.env src/demo.js", "relay": "node --env-file=.env src/relay.js", "consumer": "node --env-file=.env src/consumer.js" }, "dependencies": { "pg": "^8.16.3" } } Install dependencies with npm install. Create .env and set the connection details; keep the file out of version control:
.env
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/outbox_demo CONSUMER_URL=http://localhost:4000/events Create the database, then apply the schema with psql "$DATABASE_URL" -f schema.sql. The snippets below assume src/ exists and the schema has already been applied.
Create src/db.js:
src/db.js
import pg from "pg"; export const pool = new pg.Pool({ connectionString: process.env.DATABASE_URL, max: 10, connectionTimeoutMillis: 2_000, idleTimeoutMillis: 30_000, }); pool.on("error", (error) => { console.error("Unexpected idle PostgreSQL client error", error); }); 3. Write the order and outbox event together
The transaction must use one checked-out pg client for every statement. node-postgres explicitly warns against using pool.query() for a transaction because the pool can dispatch separate statements to different clients. node-postgres transaction documentation.
Create src/create-order.js:
src/create-order.js
import { pool } from "./db.js"; export async function createOrder({ customerId, totalCents }) { const client = await pool.connect(); try { await client.query("BEGIN"); const orderResult = await client.query( `INSERT INTO orders (customer_id, total_cents) VALUES ($1, $2) RETURNING id, customer_id, total_cents, created_at`, [customerId, totalCents], ); const order = orderResult.rows[0]; const event = { orderId: order.id, customerId: order.customer_id, totalCents: order.total_cents, createdAt: order.created_at, }; await client.query( `INSERT INTO outbox_events (aggregate_type, aggregate_id, event_type, payload) VALUES ($1, $2, $3, $4::jsonb)`, ["order", order.id, "order.created", JSON.stringify(event)], ); await client.query("COMMIT"); return order; } catch (error) { await client.query("ROLLBACK"); throw error; } finally { client.release(); } } The event payload is constructed from the row returned by the insert, not from untrusted request fields alone. The transaction ensures the order and event row either both commit or both roll back. PostgreSQL documents this all-or-nothing behavior for transaction blocks. PostgreSQL transaction tutorial.
Add src/demo.js to create one order and populate the outbox:
src/demo.js
import { randomUUID } from "node:crypto"; import { createOrder } from "./create-order.js"; import { pool } from "./db.js"; try { const order = await createOrder({ customerId: randomUUID(), totalCents: 2_599, }); console.log("Created order:", order.id); } finally { await pool.end(); } Run the consumer in one terminal with npm run consumer, the relay in another with npm run relay, then run npm run demo. The generated customer UUID is illustrative because this minimal schema does not model a customers table.
4. Poll, claim, publish, and retry
The relay must support more than one worker without letting them claim the same pending row at the same time. PostgreSQL's FOR UPDATE SKIP LOCKED is intended for queue-like tables where concurrent workers should skip rows already locked by another worker. PostgreSQL SELECT\ documentation.
The claim operation below uses a short transaction to lease a batch, then commits before making network calls. It also prevents a later unpublished event for the same aggregate from being claimed while an earlier one remains pending. This preserves per-aggregate progression in this polling example; it does not establish a global order across all aggregates.
Create src/relay.js:
src/relay.js
import { randomUUID } from "node:crypto"; import { setTimeout as sleep } from "node:timers/promises"; import { pool } from "./db.js"; const workerId = randomUUID(); const endpoint = process.env.CONSUMER_URL ?? "http://localhost:4000/events"; const batchSize = 20; const leaseSeconds = 30; async function claimBatch() { const client = await pool.connect(); try { await client.query("BEGIN"); const result = await client.query( `WITH candidates AS ( SELECT e.id FROM outbox_events e WHERE e.published_at IS NULL AND e.available_at <= now() AND (e.locked_until IS NULL OR e.locked_until < now()) AND NOT EXISTS ( SELECT 1 FROM outbox_events earlier WHERE earlier.aggregate_type = e.aggregate_type AND earlier.aggregate_id = e.aggregate_id AND earlier.published_at IS NULL AND (earlier.created_at, earlier.id) < (e.created_at, e.id) ) ORDER BY e.available_at, e.created_at, e.id FOR UPDATE SKIP LOCKED LIMIT $1 ) UPDATE outbox_events e SET locked_by = $2, locked_until = now() + ($3 * interval '1 second'), attempts = attempts + 1 FROM candidates c WHERE e.id = c.id RETURNING e.id, e.aggregate_type, e.aggregate_id, e.event_type, e.payload, e.attempts`, [batchSize, workerId, leaseSeconds], ); await client.query("COMMIT"); return result.rows; } catch (error) { await client.query("ROLLBACK"); throw error; } finally { client.release(); } } async function markPublished(eventId) { await pool.query( `UPDATE outbox_events SET published_at = now(), locked_by = NULL, locked_until = NULL, last_error = NULL WHERE id = $1 AND locked_by = $2`, [eventId, workerId], ); } async function scheduleRetry(event, error) { const delayMs = Math.min(300_000, 1_000 * 2 ** Math.min(event.attempts, 8)); await pool.query( `UPDATE outbox_events SET available_at = now() + ($2 * interval '1 millisecond'), locked_by = NULL, locked_until = NULL, last_error = $3 WHERE id = $1 AND locked_by = $4`, [event.id, delayMs, String(error).slice(0, 2_000), workerId], ); } async function deliver(event) { const response = await fetch(endpoint, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ eventId: event.id, aggregateType: event.aggregate_type, aggregateId: event.aggregate_id, type: event.event_type, payload: event.payload, }), signal: AbortSignal.timeout(5_000), }); if (!response.ok) { throw new Error(`Consumer returned HTTP ${response.status}`); } } async function run() { for (;;) { const events = await claimBatch(); if (events.length === 0) { await sleep(1_000); continue; } for (const event of events) { try { await deliver(event); await markPublished(event.id); } catch (error) { console.error("Outbox delivery failed", event.id, error); await scheduleRetry(event, error); } } } } run().catch((error) => { console.error("Relay stopped", error); process.exitCode = 1; }).finally(async () => { await pool.end(); }); A lease lets another worker recover a row if the process dies before it releases the claim. Set the lease longer than the normal delivery timeout, and renew it if a delivery can legitimately take longer. The sample uses sequential delivery within a batch to keep the flow easy to follow; increase throughput deliberately, while retaining the per-aggregate ordering rule.
5. Make the consumer idempotent
The relay marks a row as published only after receiving a successful HTTP response. Consider the failure window where the consumer commits its database transaction, but the relay crashes before updating published_at. The row becomes eligible again after its lease expires. The consumer must recognize the repeated event ID.
Create src/consumer.js:
src/consumer.js
import { createServer } from "node:http"; import { pool } from "./db.js"; const server = createServer(async (req, res) => { if (req.method !== "POST" || req.url !== "/events") { res.writeHead(404).end(); return; } try { let raw = ""; for await (const chunk of req) { raw += chunk; if (raw.length > 1_000_000) throw new Error("Request too large"); } const event = JSON.parse(raw); if (!event.eventId || event.type !== "order.created") { res.writeHead(400).end("Invalid event"); return; } const client = await pool.connect(); try { await client.query("BEGIN"); const inserted = await client.query( `INSERT INTO consumer_inbox (event_id) VALUES ($1) ON CONFLICT (event_id) DO NOTHING RETURNING event_id`, [event.eventId], ); if (inserted.rowCount === 0) { await client.query("COMMIT"); res.writeHead(200).end("Duplicate ignored"); return; } await client.query( `INSERT INTO received_order_events (event_id, order_id, payload) VALUES ($1, $2, $3::jsonb)`, [event.eventId, event.payload.orderId, JSON.stringify(event.payload)], ); await client.query("COMMIT"); res.writeHead(200).end("Accepted"); } catch (error) { await client.query("ROLLBACK"); throw error; } finally { client.release(); } } catch (error) { console.error("Consumer request failed", error); res.writeHead(500).end("Retry later"); } }); server.listen(4000, () => console.log("Consumer listening on :4000")); The inbox insert and local side effect share one consumer transaction. A duplicate event ID causes the insert to return no row, so the handler skips the side effect and responds successfully. PostgreSQL's ON CONFLICT DO NOTHING provides the uniqueness-based guard used here. PostgreSQL INSERT\ documentation.
The demo creates an order, the relay delivers its outbox event, and the consumer records it in received_order_events. The code is a runnable teaching example, not a tested deployment; add authentication, request validation, graceful shutdown, structured logging, and separate database credentials before production use.
Polling versus change data capture
Polling is the approach above: a worker queries the outbox on an interval, claims rows, sends them, and updates their status.
- It is straightforward to operate with an ordinary database and application process.
- You control batch size, retry policy, leases, and cleanup.
- It adds periodic queries and creates a delay between commit and the next poll.
- The relay owns more delivery-state logic, including leases, retries, and poison-event handling.
Change data capture (CDC) reads database change records and streams inserts from the outbox to a downstream transport. Debezium's Outbox Event Router, for example, is designed to capture an outbox table and transform its rows into routed event messages. Its documented table model includes an event ID, aggregate type, aggregate ID, event type, and payload. Debezium Outbox Event Router.
CDC can avoid frequent polling queries and can integrate with an established log-based data pipeline. It also introduces connector deployment, database-log configuration and retention, schema mapping, and connector monitoring. Choose it when the organization already operates CDC infrastructure or needs its streaming characteristics. Polling is often the smaller first implementation. Neither option removes the need for idempotent consumers.
Delivery semantics, duplicates, and ordering
Treat the relay as at-least-once: an event can be delivered more than once. This is a property of the retry workflow, not a promise that every transport behaves identically. The consumer must be safe when the same event ID arrives again.
The inbox pattern works when the inbox record and the consumer's database-side effect commit in the same transaction. If the side effect is an external ...