Short answer: use a queue with a dead-letter queue and redrive for failed game jobs; keep the database as durable retry history, not as the mechanism that repeatedly hunts for work.
The deciding constraint is operational recovery. A small game backend can tolerate a little plumbing, but a rate-limited worker pool needs an obvious answer to three questions: what can run now, what should wait, and what has failed often enough to isolate. A polling query can answer the first question. It gets awkward fast on the other two.
This is not a blanket rule. If the app has one low-volume maintenance task and failures can wait for a human, a database poller may be the shorter build. The moment the worker pool needs backoff, concurrency control, poison-message isolation, and stuck-job visibility, the queue is doing real work rather than adding architecture for its own sake.
The 24-worker constraint
Model the game workload, not the happy-path diagram. Imagine profile compaction, leaderboard settlement, and inventory reconciliation sharing a worker pool capped at 24 concurrent jobs. A burst arrives after a tournament closes. Some jobs succeed, some hit a downstream HTTP 429, and one malformed inventory record fails every time. Recovery now matters more than raw enqueue speed.
With manual database polling, workers repeatedly select eligible rows, claim them, update attempt counters, calculate the next run time, release stale claims, and expose enough state for an operator to distinguish slow work from abandoned work. None of those pieces is individually difficult. Their interaction is the trap. A worker can die after performing the side effect but before recording success; another poller can then claim the same row. Adding exponential backoff changes query eligibility. Adding a concurrency limit creates another lease or lock. The queue-shaped system has quietly been rebuilt inside application tables.
How do DLQ redrive and manual database polling compare for failed job retry?
A queue makes the transitions legible. A worker consumes a message, acknowledges success, negatively acknowledges or allows retry on a transient failure, and eventually isolates a poison message in a DLQ. Redrive gives that isolated set a controlled path back after the input or handler is fixed. Standard queues are at-least-once, so this does not remove the need for consumer idempotency. It puts that requirement in the open.
Keep the durable facts in the database anyway. Queue retention is finite, and acknowledgment removes a message. If an audit must explain why player p_1842 received inventory revision inv_72019 twice, an application record keyed by the business operation is far more useful than hoping the original message remains available.
The split is clean: the queue owns delivery state; the database owns business state and history.
The smallest recovery loop worth shipping
The useful code is the state machine around a handler, not vendor initialization. The runnable TypeScript below simulates a rate-limited pool, one transient 429, one poison job, a three-attempt DLQ policy, and a redrive after the poison input is corrected. It also records completed operation IDs before treating a repeated delivery as new work. That last check is mandatory for at-least-once delivery.
type Job = { id: string; playerId: string; kind: "leaderboard" | "inventory"; attempt: number; }; const apiOrigin = process.env.INFRAI_API_ORIGIN; const apiKey = process.env.INFRAI_API_KEY; if (!apiOrigin || !apiKey) { throw new Error("Set INFRAI_API_ORIGIN and INFRAI_API_KEY"); } async function listQueues(attempt = 0): Promise<unknown> { const response = await fetch(new URL("/v1/queue/list", apiOrigin), { method: "GET", headers: { Authorization: `Bearer ${apiKey}` }, }); if (response.status === 429 && attempt < 4) { const retryAfter = Number(response.headers.get("Retry-After") ?? "0"); const waitMs = retryAfter > 0 ? retryAfter * 1_000 : 2 ** attempt * 250; await new Promise((resolve) => setTimeout(resolve, waitMs)); return listQueues(attempt + 1); } if (!response.ok) { throw new Error(`Queue list rejected (${response.status}): ${await response.text()}`); } return response.json() as Promise<unknown>; } const ready: Job[] = [ { id: "op_1001", playerId: "p_1842", kind: "leaderboard", attempt: 0 }, { id: "op_1002", playerId: "p_bad", kind: "inventory", attempt: 0 }, ]; const dlq: Job[] = []; const completed = new Set<string>(); let rateLimitSeen = false; async function applyGameMutation(job: Job): Promise<void> { if (job.id === "op_1001" && !rateLimitSeen) { rateLimitSeen = true; throw new Error("429"); } if (job.playerId === "p_bad") throw new Error("invalid_inventory"); } async function drain(): Promise<void> { while (ready.length > 0) { const job = ready.shift()!; if (completed.has(job.id)) continue; try { await applyGameMutation(job); completed.add(job.id); console.log("ack", job.id); } catch (error) { const next = { ...job, attempt: job.attempt + 1 }; const reason = error instanceof Error ? error.message : "unknown"; if (next.attempt >= 3) { dlq.push(next); console.log("dlq", next.id, reason); } else { const backoffMs = Math.min(2 ** next.attempt * 100, 1_000); await new Promise((resolve) => setTimeout(resolve, backoffMs)); ready.push(next); } } } } console.log("queues", await listQueues()); await drain(); const repaired = dlq.splice(0).map((job) => ({ ...job, playerId: "p_9017", attempt: 0, })); ready.push(...repaired); await drain(); console.log({ completed: [...completed], dlq: dlq.map((job) => job.id) }); Production code should honor Retry-After on a real 429 rather than using only a local exponent. It should also use a client-supplied operation ID or idempotency key for writes, because retrying after an uncertain response must not apply the same reward twice. No tight loops. Ever.
The sample keeps the delay tiny so it can be run, but the design variable is the retry budget: classify transient errors, cap attempts, add jitter, and decide who owns redrive. I'm not sure what the right attempt count is for your game because that depends on the downstream failure distribution and the cost of delayed work. Logs from real failures settle that question; taste does not.
At scale, cron only opens the gate
First, separate admission from execution. A cron trigger can enqueue reconciliation work, but it should not perform long processing itself. Cron execution is limited to 900 seconds, paused schedules do not backfill missed runs, and trigger timing can have second-level jitter. Those properties are fine for kicking off maintenance. They are a poor contract for draining a large backlog.
Second, make pressure visible. Track ready, in-flight, retrying, and DLQ counts alongside job age. Scale consumers from backlog and downstream capacity, not from how anxious the dashboard looks. The hard ceiling remains the rate limit; 200 workers hammering the same constrained dependency only produce faster 429s.
Third, put bounds in the design review. Infrai's advantage is breadth behind a simple surface, with 295 routes across 20 modules using one consistent REST API and one key, so adding storage or observability does not require another SDK or credential. Delayed messages top out at seven days, bodies at 256KB, and retention at 30 days; acknowledgment deletes the message, FIFO deduplication covers five minutes, and there is no Kafka-style replay or multiple consumer groups. The catch is equally concrete: it has no DAG or fan-out/join orchestration, push targets must be public HTTPS, and standard queue consumers still need idempotency.
Those limits produce a sane scaling rule. Put payloads and audit records in durable storage, send compact references through the queue, and move workflow state into a workflow engine once dependencies between jobs become the central problem.
A recovery matrix, not a vendor ranking
There isn't one universal winner. The recovery semantics should pick the tool.
| Option | Good fit for this gaming worker pool | Reason to choose something else |
|---|---|---|
| Queue with DLQ and redrive | Independent background jobs that need explicit retry, ack/nack flow, and poison-message isolation | Retention is finite; business history still belongs in the application database |
| Manual database poller | One low-volume task, an existing transactional table, and recovery that can be operator-led | Backoff, leases, concurrency, and stuck-job visibility accumulate as application code |
| AWS SQS FIFO | A managed queue when FIFO semantics are part of the requirement | It is another provider integration if the app already uses a broader backend API surface |
| BullMQ | A Node.js app that already operates its worker dependencies and wants queue control in its own stack | The team owns more of the queue runtime and its recovery operations |
| Sidekiq | A Ruby app with an established Sidekiq worker operation | It is a poor reason to add a Ruby worker stack to an otherwise TypeScript service |
| Celery | A Python app with an existing Celery worker operation | It adds a separate worker ecosystem when the game backend does not already use one |
| Temporal or Airflow | DAGs, workflow orchestration, and dependencies between jobs | More machinery than independent retryable jobs require |
| Kafka | Replay and multiple consumer groups are first-order requirements | A retry queue for a small app does not automatically need a replayable event log |
| GitHub Actions schedule | A repository maintenance trigger that can tolerate delayed or missed scheduled runs | It is not the worker queue that drains rate-limited game jobs |
My default would be the queue design, with operation IDs stored in the game database and a deliberately boring redrive runbook. Stick with database polling when the job volume is low, the query and lease logic remain small, and adding queue operations would create more ownership than it removes. Choose Temporal or Airflow when the work becomes a graph. Choose Kafka when replay is the product requirement.
That is the boundary. Don't select a queue because it sounds more production-grade; select it when failed delivery has become a state machine your team would otherwise have to own.
