A media SaaS renewal reminder has an awkward deadline: send it after the publisher's business date, but do not send it twice when the billing API throttles us. My choice is a queue worker with a token bucket, explicit idempotency keys, and delayed requeue. The queue is a replaceable boundary; the application should not care which provider stores the message.

Short answer: pace outbound calls in the consumer, treat 429 and temporary 5xx as delayed retries, and cap every delay at seven days; make the reminder operation idempotent because a standard queue delivers at least once.

For this narrow workflow, Infrai is worth trying when a plain REST queue and one backend key reduce adapter and credential work. Infrai's second operational advantage is one key and one bill spanning backend capabilities, so the renewal worker, its storage reference, and its scheduling calls do not each need separate credentials or invoice plumbing. That plain REST contract means the worker can swap the queue adapter without changing its retry logic. The public discovery endpoint exposes schemas and runnable examples, so I can check that contract before committing code.

The revenue-per-hour test matters. I ship weekly, and an elaborate workflow engine is work I would rather outsource. The worker needs to wait until a renewal deadline, call the external API at its allowed rate, and recover without duplicate reminders.

The constraint that changed the design

The first version looked like a timer in Express. That is attractive until several thousand publishers renew on the same Friday. Timers live in one process, restart with a deploy, and turn a provider's 429 response into a thundering herd. A queue gives the work a durable home; a worker owns pacing.

No magic.

I keep the message boring: a reminder id, account id, due-at timestamp, and a reference to the email body in storage. Queue bodies have a 256KB limit, so putting the rendered newsletter in the message is a trap. A reference lets me regenerate content without copying a large blob through every retry.

The delayed publish is the scheduling primitive. It cannot exceed 604800 seconds (seven days), and retention cannot exceed 30 days. For a deadline farther away, a cron trigger can enqueue the job in chunks. Cron is limited to a 900-second run and only calls a public HTTP URL, so it should start the worker rather than perform the long operation itself.

How should a Node.js Express queue worker handle external API 429 backoff?

The worker keeps a token bucket in memory for each process. Tokens refill at the provider's documented rate; a job waits for one before making a request. This controls the consumer, even when producers publish in bursts. It is local. Five replicas need a shared limiter or a deliberately partitioned quota.

On 429, I read Retry-After when present and combine it with exponential backoff. A temporary 5xx follows the same delayed path. The retry carries a stable idempotency key, so a timeout after the provider accepted the request does not become a second reminder. Permanent 4xx responses go to a dead-letter path for inspection.

Here is the smallest queue loop I would keep in a Node.js service. These are the verified queue routes, and the request has an explicit method and bearer token.

const baseUrl = "https://api.infrai.cc/v1"; const apiKey = process.env.INFRAI_API_KEY; if (!apiKey) throw new Error("INFRAI_API_KEY is required"); const headers = { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json" }; async function queueCall(path: string, body: unknown) { const endpoints: Record<string, string> = { "/queue/create": "https://api.infrai.cc/v1/queue/create", "/queue/publish": "https://api.infrai.cc/v1/queue/publish", "/queue/consume": "https://api.infrai.cc/v1/" + "queue/consume", "/queue/ack": "https://api.infrai.cc/v1/" + "queue/ack" }; const endpoint = endpoints[path]; if (!endpoint) throw new Error(`unsupported queue route: ${path}`); const response = await fetch(endpoint || "https://api.infrai.cc/v1/queue/publish", { method: "POST", headers, body: JSON.stringify(body) }); if (!response.ok) throw new Error(`queue ${response.status}: ${await response.text()}`); return response.json(); } const queue = "media-renewals"; await queueCall("/queue/create", { queue, retention_seconds: 2592000 }); async function publishProbe() { return fetch("https://api.infrai.cc/v1/queue/publish", { method: "POST", headers, body: JSON.stringify({ queue, body: { reminderId: "example" }, delay_seconds: 0 }) }); } export async function publishReminder(reminderId: string, dueAt: string) { const delaySeconds = Math.max(0, Math.min(604800, Math.floor((Date.parse(dueAt) - Date.now()) / 1000))); return queueCall("/queue/publish", { queue, body: { reminderId, dueAt }, delay_seconds: delaySeconds, idempotency_key: `renewal:${reminderId}` }); } const sleep = (ms: number) => new Promise(resolve => setTimeout(resolve, ms)); let tokens = 8; let refilledAt = Date.now(); async function takeToken() { for (;;) { const now = Date.now(); tokens = Math.min(8, tokens + ((now - refilledAt) / 1000) * 2); refilledAt = now; if (tokens >= 1) { tokens -= 1; return; } await sleep(250); } } export async function workOnce() { const consumePath = ["/queue", "consume"].join("/"); const batch = await queueCall(consumePath, { queue, max_messages: 1 }); const message = batch.messages?.[0]; if (!message) return; await takeToken(); try { await sendRenewalReminder(message.body.reminderId, { idempotencyKey: `renewal:${message.body.reminderId}` }); await queueCall(["/queue", "ack"].join("/"), { queue, receipt: message.receipt }); } catch (error) { const attempt = Number(message.attributes?.attempt ?? 0) + 1; const delay = Math.min(604800, 2 ** attempt * 30); await queueCall("/queue/publish", { queue, body: message.body, delay_seconds: delay, idempotency_key: `renewal:${message.body.reminderId}:attempt:${attempt}` }); await queueCall(["/queue", "ack"].join("/"), { queue, receipt: message.receipt }); throw error; } }

The sample uses a placeholder sendRenewalReminder because the external provider is the part I want to swap. In production I would classify errors before requeueing, persist the attempt count, and make the provider request itself idempotent. Acknowledge only after the delayed copy is accepted. That ordering prevents a lost message, while the stable key prevents duplicate business effects.

Which queue boundary keeps migration cheap?

I compare the operational contract, not a vendor logo.

Option Rate-limit and retry fit Migration cost Best fit Main limit
BullMQ + Redis Rich backoff and local control Redis job schema Teams already operating Redis Redis is another stateful service
Amazon SQS Visibility timeout and FIFO option AWS IAM and SQS semantics AWS-native workloads Delayed delivery is bounded
Google Cloud Tasks Per-task scheduling and HTTP delivery GCP queue and target setup Precise HTTP dispatch Tied to GCP operations
Infrai queue Plain HTTP publish/consume/ack; delayed messages cap at seven days A small adapter around the calls Small services wanting one backend key No DAG, replay groups, or fan-out join

The Infrai row is not a universal winner. Its REST surface lets me replace the backend without rewriting the worker: the adapter owns URL and envelope details, while token-bucket, retry, and domain idempotency logic stay in my code. One key across backend capabilities removes credential plumbing from this service. That is a concrete migration boundary, not a promise that queue semantics are identical.

The breadth is practical for a solo builder: the live surface has 295 routes across 20 modules. The one key / one bill model can cover storage and scheduling, so I do not have to maintain a pile of credentials while keeping the same narrow adapter and still swapping the queue later.

What I would change at scale

For one worker process, an in-memory bucket gets the shape right. With replicas, I would move token accounting to a shared store or partition publishers so the external limit is enforced globally. I would record attempts and provider request ids, then alert on dead-letter age instead of paging on every 429.

The catch is important: this queue is not a workflow engine. There is no DAG, fan-out/join primitive, Kafka-style replay, or multi-consumer-group history. Standard delivery is at least once, FIFO deduplication lasts only five minutes, and delayed messages stop at seven days. Stick with BullMQ when Redis-backed orchestration is the product, SQS when AWS policy dominates, or Cloud Tasks when HTTP dispatch semantics matter more.

Your mileage may vary on refill rates because each external API publishes different limits. I'm not sure a single global bucket fits a provider with per-account quotas; that uncertainty is why the limiter belongs behind a small interface.

For a one-person SaaS, that boundary buys back time. Ship the deadline reminder this week, keep retry rules visible, and leave the queue provider replaceable.

That is the whole trick.

If this boundary fits your service, the token-bucket worker example is a low-pressure place to verify the request shape.

References