A retry budget defines how many retries a client may afford in total, no matter how many attempts a single request would still have left. That sounds like bookkeeping, but it is the difference between “the partner had a brief hiccup” and “we knocked the partner over for good with our own retries”. In my article on exponential backoff with jitter, the question was when to retry. Today the question is whether you should retry at all.
The topic keeps me busy because retries feel so harmless. Three attempts, a bit of waiting, done. But once the target really falls over, every client turns into a small load generator. That is exactly when a budget helps, because it makes retries scarce as soon as too many requests fail.
In short
A retry budget limits the sum of all retries of a client, not just the attempts per request. Retries stay allowed for occasional glitches but become scarce quickly during a real outage. The biggest pitfall in Lambda: every execution environment has its own budget, and Lambda scales out exactly when things get tight.
What you will learn in this article
- What a retry budget is and why backoff alone is not enough
- Token bucket like the AWS SDK versus a 10% ratio like Google SRE
- A tested retry budget in TypeScript with your own retry loop
- How to use the budget in Lambda and where it hits its limits there
What is a retry budget?
A retry budget is an allowance for retries that all requests of a client share. The well-known limit of “at most three attempts” applies per request. The budget, on the other hand, applies to the whole process: as long as almost everything works, there is plenty left, and occasional outliers may try again. If many requests suddenly fail, the budget is used up fast. From then on, the client passes errors on immediately instead of piling on.
The idea behind it is simple. A single error is usually chance, for example a lost packet or a container that is just restarting. Many errors at once are almost never chance. In that case the target is overloaded or broken, and every further attempt makes the problem worse. That is why Marc Brooker calls retries “selfish” in the Amazon Builders’ Library: the retrying client buys better odds with the server’s time.
Why backoff alone is not enough
Exponential backoff with jitter spreads retries over time. That prevents all clients from knocking again at the same moment. However, backoff does not change the number of attempts. If every request has three attempts and all of them fail, almost three times as many calls still reach the target, just stretched out a little.
The Google SRE book works through exactly this case: with a limit of three attempts per request, the load ends up just below three times the original in the worst case. Add a per-client ratio that caps retries at 10 percent of requests, and the increase drops to about 1.1. On top of that comes the multiplication across layers, which I already calculated in the backoff article. A budget slows down each of those layers.
Two ways to build a retry budget
In practice you will meet two variants. Both work, yet they feel different.
Token bucket like in the AWS SDK
In standard mode, the AWS SDK uses a bucket of tokens. According to the current SDK reference, it holds 500 tokens. A retry after a transient error such as a timeout or a 5xx response costs 14 tokens, and a retry after throttling (HTTP 429) costs 5. Every success on the first attempt puts one token back, and a successful retry gets its own cost refunded. Once the bucket is empty, the SDK stops retrying and passes the error through.
Important context: these values belong to the new retry behavior AWS announced for 2026. Until it becomes the default, you switch it on with the environment variable AWS_NEW_RETRIES_2026=true. For the JavaScript SDK v3, the tracking issue names November 2026 as the earliest switch-over. The idea of the bucket, though, has been in the SDK since 2016.
Ratio like in the Google SRE book
The second variant keeps count: how many requests has this client made in the last time window, and how many of them were retries? A retry is allowed only as long as that share stays below a threshold, which is 10 percent at Google. The Envoy proxy knows the same principle as retry_budget, with 20 percent as the default and a floor of at least three concurrent retries.
This floor matters. Without it, a client with little traffic would hardly ever get a retry, because 10 percent of five requests is half a retry.
| Token bucket | Ratio | |
|---|---|---|
| Keeps track of | Fill level of a bucket | Requests and retries in the time window |
| Reacts to | Errors that pile up | Share of retries in the traffic |
| Recovers through | Successes that refill it | Time, as the window moves on |
| Strength | Allows a buffer for short error spikes | Scales with traffic automatically |
| Weakness | A fixed buffer does not fit every volume | Needs a floor for low traffic |
Result of the simulation from step 5: without a budget, the target receives roughly twice the load during the outage.
Build a retry budget in TypeScript
Now it gets practical. The example is a fictional bike rental whose Lambda function asks a partner how many bikes are available at a station. The partner is sometimes slow and sometimes overloaded, which is exactly the case where you want a retry budget.
Prerequisites
- Node.js 22 or newer (the example uses the built-in
fetchand thenode:testtest runner) - Basic TypeScript knowledge
- For the deployment: an AWS account and the Serverless Framework v4
Step 1: Create the project
mkdir retry-budget && cd retry-budget npm init -y npm install --save-dev typescript @types/node mkdir src test Next, adjust the package.json. The important part is "type": "module", so that Node loads the compiled files as ES modules. The tsconfig.json contains "types": ["node"], because TypeScript 7 no longer includes the Node types automatically.
{ "name": "retry-budget", "version": "1.0.0", "private": true, "type": "module", "scripts": { "build": "tsc", "test": "tsc && node --test dist/test/*.test.js", "simulate": "tsc && node dist/src/simulate.js" }, "devDependencies": { "@types/node": "^22.20.5", "typescript": "^7.0.2" } } { "compilerOptions": { "target": "ES2022", "module": "NodeNext", "moduleResolution": "NodeNext", "strict": true, "outDir": "dist", "rootDir": ".", "skipLibCheck": true, "types": ["node"] }, "include": ["src", "test"] } Step 2: The two budgets
Both variants get the same interface. That way you can swap them later without touching the retry loop. The method tryAcquireRetry() is the core: before every retry it asks “may I?” and books the attempt if the answer is yes.
// src/retry-budget.ts export type RetryKind = "transient" | "throttling"; /** Shared interface: may I try once more right now? */ export interface RetryBudget { /** Call before every first attempt. */ recordRequest(): void; /** Call before every retry. false = budget empty, pass the error on. */ tryAcquireRetry(kind?: RetryKind): boolean; /** Call after every success; lastRetry = kind of the last retry, if there was one. */ recordSuccess(lastRetry?: RetryKind): void; } export interface TokenBudgetOptions { capacity?: number; // size of the bucket transientCost?: number; // cost of a retry after a timeout or 5xx throttlingCost?: number; // cost of a retry after 429 successRefund?: number; // refund for a success on the first attempt } /** * Token bucket like in the AWS SDK: retries cost tokens, successes refill. * Once the bucket is empty, nothing is retried any more. */ export class TokenRetryBudget implements RetryBudget { private tokens: number; private readonly capacity: number; private readonly transientCost: number; private readonly throttlingCost: number; private readonly successRefund: number; constructor(options: TokenBudgetOptions = {}) { this.capacity = options.capacity ?? 500; this.transientCost = options.transientCost ?? 14; this.throttlingCost = options.throttlingCost ?? 5; this.successRefund = options.successRefund ?? 1; this.tokens = this.capacity; } recordRequest(): void { // The first attempt is free. } private cost(kind: RetryKind): number { return kind === "throttling" ? this.throttlingCost : this.transientCost; } tryAcquireRetry(kind: RetryKind = "transient"): boolean { const cost = this.cost(kind); if (this.tokens < cost) return false; this.tokens -= cost; return true; } recordSuccess(lastRetry?: RetryKind): void { // Successful retry: refund only its own cost, otherwise the small refund. const refund = lastRetry ? this.cost(lastRetry) : this.successRefund; this.tokens = Math.min(this.capacity, this.tokens + refund); } get available(): number { return this.tokens; } } export interface RatioBudgetOptions { ratio?: number; // share of retries relative to requests, e.g. 0.1 = 10% windowMs?: number; // observation window minRetries?: number; // floor so that low traffic still gets retries now?: () => number; // replaceable for tests } /** * Ratio budget like in the Google SRE book: retries may make up at most * a fixed share of the requests in the sliding window. */ export class RatioRetryBudget implements RetryBudget { private readonly ratio: number; private readonly windowMs: number; private readonly minRetries: number; private readonly now: () => number; private readonly slotMs: number; private readonly slots: { start: number; requests: number; retries: number }[] = []; constructor(options: RatioBudgetOptions = {}) { this.ratio = options.ratio ?? 0.1; this.windowMs = options.windowMs ?? 10_000; this.minRetries = options.minRetries ?? 3; this.now = options.now ?? Date.now; this.slotMs = Math.max(1, Math.floor(this.windowMs / 10)); } private current() { const t = this.now(); const start = t - (t % this.slotMs); // Drop old slots that have fallen out of the window. while (this.slots.length > 0 && this.slots[0].start <= t - this.windowMs) { this.slots.shift(); } let slot = this.slots[this.slots.length - 1]; if (!slot || slot.start !== start) { slot = { start, requests: 0, retries: 0 }; this.slots.push(slot); } return slot; } recordRequest(): void { this.current().requests++; } tryAcquireRetry(): boolean { const slot = this.current(); const requests = this.slots.reduce((sum, s) => sum + s.requests, 0); const retries = this.slots.reduce((sum, s) => sum + s.retries, 0); const allowed = Math.max(this.minRetries, requests * this.ratio); if (retries + 1 > allowed) return false; slot.retries++; return true; } recordSuccess(): void { // The ratio budget needs no refund. } } For the ratio budget, I split the window into ten slots. That gives you a sliding window with coarse resolution: old slots drop out at the back without you having to store every single request with a timestamp.
Step 3: The retry loop
The loop decides in this order: is the error retryable at all? Does this request have attempts left? Does the budget grant a retry? Only then does it wait with full jitter and try again. If the budget is empty, it throws its own error type. You can catch that further up on purpose.
// src/with-retries.ts import { setTimeout as sleep } from "node:timers/promises"; import type { RetryBudget, RetryKind } from "./retry-budget.js"; /** Error with an HTTP status so we can tell retryable from final failures. */ export class HttpError extends Error { constructor(public readonly status: number, message: string) { super(message); this.name = "HttpError"; } } export class RetryBudgetExhaustedError extends Error { constructor(public readonly cause: unknown, public readonly attempts: number) { super(`Retry budget exhausted after ${attempts} attempt(s)`); this.name = "RetryBudgetExhaustedError"; } } export interface RetryOptions { budget: RetryBudget; maxAttempts?: number; // including the first attempt baseDelayMs?: number; maxDelayMs?: number; signal?: AbortSignal; random?: () => number; // replaceable for tests } /** Classifies an error: null = do not retry. */ export function classify(error: unknown): RetryKind | null { if (error instanceof HttpError) { if (error.status === 429) return "throttling"; if ([500, 502, 503, 504].includes(error.status)) return "transient"; return null; // 4xx except 429: a second attempt changes nothing } if (error instanceof Error && (error.name === "TimeoutError" || error.name === "TypeError")) { return "transient"; // timeout or network error from fetch } return null; } export async function withRetries<T>( operation: (attempt: number) => Promise<T>, options: RetryOptions, ): Promise<T> { const { budget, maxAttempts = 3, baseDelayMs = 50, maxDelayMs = 2_000, signal } = options; const random = options.random ?? Math.random; let lastRetry: RetryKind | undefined; budget.recordRequest(); for (let attempt = 1; ; attempt++) { try { const result = await operation(attempt); budget.recordSuccess(lastRetry); return result; } catch (error) { const kind = classify(error); if (kind === null || attempt >= maxAttempts) throw error; if (!budget.tryAcquireRetry(kind)) throw new RetryBudgetExhaustedError(error, attempt); lastRetry = kind; // Full jitter: random wait between 0 and the exponential ceiling. const ceiling = Math.min(maxDelayMs, baseDelayMs * 2 ** (attempt - 1)); await sleep(Math.floor(random() * ceiling), undefined, { signal }); } } } The loop never retries errors with status 400, 401 or 404. A second attempt with the same broken request achieves nothing, it only costs budget.
🎬 Request flow with a retry budget – this animated diagram only runs on my site: watch it on sascha-ms.de
Step 4: Test
The tests check the behavior that matters: the bucket runs dry, the ratio sticks to 10 percent, a 503 is retried and a 400 is not. The time source is replaceable, so no test has to wait for real seconds.
// test/retry-budget.test.ts import { test } from "node:test"; import assert from "node:assert/strict"; import { RatioRetryBudget, TokenRetryBudget } from "../src/retry-budget.js"; import { HttpError, RetryBudgetExhaustedError, withRetries } from "../src/with-retries.js"; test("token bucket: empty bucket blocks further retries", () => { const budget = new TokenRetryBudget({ capacity: 30, transientCost: 14 }); assert.equal(budget.tryAcquireRetry("transient"), true); assert.equal(budget.tryAcquireRetry("transient"), true); assert.equal(budget.tryAcquireRetry("transient"), false); // only 2 tokens left budget.recordSuccess("transient"); // successful retry refunds 14 assert.equal(budget.available, 16); }); test("ratio: at most 10% retries, at least minRetries", () => { let now = 0; const budget = new RatioRetryBudget({ ratio: 0.1, windowMs: 10_000, minRetries: 3, now: () => now }); for (let i = 0; i < 50; i++) budget.recordRequest(); const granted = Array.Source：https://dev.to/sascha-ms/retry-budget-in-typescript-stop-retry-storms-1fk0Related posts
I kept shipping agents I had only tested on the happy path
Teardown packages 3D machine simulators for Claude Code
Typography tokens: modeling your type scale so developers stop guessing
AI agents don't see your page. They read its accessibility tree.
Day 25/90: How a C# Event Leaked 720 MB (Delegates and Events, Properly)
