Every tutorial for scheduled work has the same shape. A cron expression, a route handler, a loop over users, a call to an email API. It runs on the author's machine for two days and then it ships.
Then production happens. The platform retries a 502 that actually succeeded. A deploy lands mid-run and two workers process the same batch. The retry after a rate limit sends the digest a second time. A customer forwards the duplicate to support, and someone writes a ticket titled "we got this twice," and that is the first time anybody thinks about delivery semantics.
Cron is a trigger, not a scheduler. The moment your work leaves the process, you are in at-least-once territory, and at-least-once means duplicates are not an edge case. They are the contract.
Make the duplicate impossible, not unlikely
The fix is not a sent_at column you check before sending, because check-then-send has a race and you will find it at 3 a.m. with two workers that both read null.
The fix is a unique constraint that decides the winner atomically, before any side effect:
import pg from "pg"; const pool = new pg.Pool({ connectionString: process.env.DATABASE_URL }); async function claim(eventId) { const { rowCount } = await pool.query( `insert into processed_events (event_id, claimed_at) values ($1, now()) on conflict (event_id) do nothing`, [eventId] ); return rowCount === 1; // true only for the first caller } if (!(await claim(event.id))) return; // someone else has it, or we already did Two details matter more than the code. The key must come from the event, not from a timestamp or a random UUID you generate at send time, and it must be stable across retries. If your queue redelivers with a fresh ID, the constraint does nothing for you. And on conflict do nothing returning rowCount = 0 tells you the event was already claimed, which is exactly the information you need to skip quietly instead of erroring.
For the recurring job itself, the same idea applies at the job level. A single-flight lock beats a if (running) return flag in memory, which does nothing when you run three replicas:
const { rows } = await pool.query( `select pg_try_advisory_lock(hashtext('nightly-digest')) as locked` ); if (!rows[0].locked) return; try { await runDigest(); } finally { await pool.query(`select pg_advisory_unlock(hashtext('nightly-digest'))`); } Bound the concurrency or drown
The other half of this problem is what happens when your provider pushes back. A loop over ten thousand users with await inside is slow but polite. A loop with Promise.all is fast and gets you a 429 within seconds, and then the retry storm makes it worse.
Bound the in-flight requests, and treat 429 as an instruction rather than an error:
import pLimit from "p-limit"; const limit = pLimit(4); // provider allows ~5 concurrent, leave headroom async function sendWithRetry(payload, attempt = 0) { try { return await limit(() => post("/v1/messages", payload)); } catch (err) { if (err.status === 429 && attempt < 5) { const retryAfter = Number(err.headers.get("retry-after") ?? 0); const backoff = retryAfter * 1000 || 2 ** attempt * 1000; await sleep(backoff + Math.random() * 250); // jitter return sendWithRetry(payload, attempt + 1); } throw err; } } Three things in there that get skipped in tutorials. Retry-After is honored when the server sends it, because the server knows better than your exponential curve. Jitter is added, because identical backoff across workers synchronizes them into a thundering herd. And there is an attempt cap, because a permanent failure should land in a dead letter queue where a human sees it, not in an infinite loop that burns your quota.
Cap it, then route it. A row in a failed_jobs table with the last error and the payload hash is worth more than any amount of clever retry logic, because it makes the failure visible instead of ambient.
Don't enqueue from inside the transaction you can still roll back
One pattern bites every team once. You write to the database and publish to the queue in the same handler. The write commits, the publish fails, and the work is lost. Or the write rolls back after the publish, and the consumer acts on a record that does not exist.
Use an outbox: write the event to a table in the same transaction as the state change, then have a separate poller publish and mark it sent. It costs one table and one query per tick, and it converts a distributed consistency problem into a local one.
create table outbox ( id bigserial primary key, topic text not null, payload jsonb not null, created_at timestamptz not null default now(), sent_at timestamptz ); create index on outbox (id) where sent_at is null; The poller claims rows with for update skip locked, which lets you run it on every replica without coordination. That is the whole trick, and it is the same trick as the idempotency key: let the database decide who wins.
The test that would have caught it
Before any of this, write the test that fails on the tutorial version of the code.
Fire the same event twice into your handler with the same event ID. Assert that exactly one email exists in the fake provider, that the second call returns 2xx, and that no error is thrown. Then add the harder one: two concurrent calls with the same ID, awaited together, asserting one send. That second test is the one that proves your constraint is doing the work instead of your application logic.
Both run in about fifty milliseconds. Neither requires a queue, a fake clock, or a staging environment.
What you actually owe the reader
The "cron plus webhook" post is not wrong because it is simple. It is wrong because it teaches a delivery model that does not exist. Nobody gets exactly-once across a network boundary, and the tutorials that skip that sentence are the reason the same digest goes out twice to a customer who has already unsubscribed once.
Write the duplicate test first. Put the unique index in before the feature ships. And when someone asks why there is a processed_events table that seems to do nothing, tell them it is the part of the system that keeps the rest of it honest.
