An S3 multipart upload is the answer as soon as files get too big for a single request. In my article on the S3 presigned URL I showed how the browser pushes files straight into a bucket without them passing through the Lambda function. That works great for profile pictures and PDFs. With a 3 GB audiobook, a video recording or a data export, things look different: a single PUT can be at most 5 GB, and if the connection drops at 90 percent, everything starts over.
This topic keeps me busy because most guides stop exactly where it gets tricky. They show how to sign one URL per part and leave open who actually controls the file size, what happens to half-finished uploads, and why the browser suddenly can’t see an ETag. So here we build the complete pattern with AWS CDK and TypeScript: four small endpoints, a DynamoDB table, a lifecycle rule, and a browser client that uploads parts in parallel and retries failures on its own.
In short
With an S3 multipart upload, the client splits a large file into parts, uploads each part through its own presigned URL straight to S3, and your backend assembles the parts at the end. The key insight: if the server signs the size of every part and counts the parts in S3 itself before completing, the client can neither cheat nor forget anything. The biggest pitfall is incomplete uploads that sit invisibly in the bucket and cost money until a lifecycle rule removes them.
What you will learn in this article
- When you need an S3 multipart upload and which limits S3 sets
- How to calculate a sensible part size and part count
- How Lambda signs fixed-size part URLs and completes the upload safely
- How a browser client uploads parts in parallel and retries errors
- Which errors are common and how a lifecycle rule prevents costs
When do you need an S3 multipart upload?
AWS recommends multipart uploads from around 100 MB. That’s a rule of thumb, not a hard limit. From this size on, the benefits become obvious. A failed part is retried on its own, several parts run in parallel and make better use of the bandwidth, and you can resume an upload later because S3 remembers every finished part. From 5 GB you no longer have a choice anyway, since a single PUT can’t be any larger.
The limits you have to plan for are manageable. These values come from the S3 documentation (as of November 2026):
| Limit | Value |
|---|---|
| Largest possible object | 48.8 TiB |
| Parts per upload | up to 10,000, numbered 1 to 10,000 |
| Size of a part | 5 MiB to 5 GiB, only the last part may be smaller |
| Parts per ListParts response | at most 1,000 |
| Single PUT without multipart | at most 5 GB |
For small files, the simple approach with a single presigned URL remains the better choice. It needs less code and fewer requests. In practice I would draw the line at 100 MB and stick with the simple PUT below that.
How an S3 multipart upload works
The flow consists of three S3 calls. With CreateMultipartUpload you register the upload and get an upload ID back. After that, you upload any number of parts with UploadPart, each with a part number. For every part, S3 returns an ETag, a kind of fingerprint of that part. Finally, CompleteMultipartUpload sends the list of all part numbers with their ETags, and only then does S3 build an object from them. Until that point, there is no file in the bucket that anyone could download.
The architecture: Lambda only manages plan and status, the bytes go straight from the browser to S3.
Here’s the trick with presigned URLs: the parts don’t have to go through your backend. The Lambda function signs a separate URL for each part, and the browser sends the bytes directly to S3. A presigned URL is a link that allows one specific S3 action for a short time, without the client needing AWS credentials of its own. Your function only handles a few small JSON requests while gigabytes flow past it.
One quirk you should know before you start: parts of an unfinished upload stay in the bucket until someone completes or aborts the upload. You pay for that storage, but you don’t see the parts as objects in the S3 console. If you forget about this, invisible gigabytes pile up over the months.
How large should the parts be?
Part size is a trade-off. Small parts mean little loss when one fails, but many requests. Large parts save requests, yet cost more time when something breaks. I start with 16 MiB. A 3 GiB file then has exactly 192 parts, and even over a mediocre connection a retried part finishes in a few seconds.
The upper limit of 10,000 parts also matters. At 16 MiB per part you get to about 156 GiB. For larger files the part size has to grow, and the planParts function further down handles that automatically: it divides the file size by 10,000, rounds up to whole MiB and takes the larger of the two values.
The architecture with four endpoints
In many examples, the client sends its collected ETags to the server at the end, and the server passes them on to S3 unchecked. That works, but it leaves two questions open: how big is the file really, and did the client upload every part? That’s why I would build it a little differently. When the upload starts, the server stores in a DynamoDB table how large the file should be and how many parts it splits into. Everything else follows from that:
- To start, POST /uploads checks file type and size, starts the multipart upload and creates a record.
- Next, POST /uploads/{id}/parts signs URLs for specific part numbers, each with the exact length of that part.
- At the end, POST /uploads/{id}/complete asks S3 via
ListPartswhich parts have arrived, compares count and sizes, and only then completes. - If something goes wrong, DELETE /uploads/{id} aborts the upload and frees the storage.
Why the server signs the part size
The decisive trick sits in the second endpoint. If you pass a ContentLength to the SDK when signing, the content-length header ends up in the signature. If the browser then sends a part with a different size, the signature no longer matches and S3 rejects the request. So the client can’t simply stuff 5 GB into every part. Together with the size check on completion, the file size is controlled by the server, even though the bytes never pass through it.
🎬 Flow of an S3 multipart upload with presigned URLs – this animated diagram only runs on my site: watch it on sascha-ms.de
The example is a fictional audiobook archive of a city library. Volunteers upload recordings there as MP3, M4B or ZIP, which quickly grow to several gigabytes. For this article I limit files to 20 GiB; that’s a rule of the app, not of S3.
Step by step: S3 multipart upload with AWS CDK
Prerequisites
- Node.js 22 or newer and npm
- An AWS account with permissions for CloudFormation, Lambda, S3, DynamoDB and API Gateway (only for deployment, the tests run without it)
- A one-time
cdk bootstrapin the target region - Basic TypeScript knowledge; I explain CDK along the way
Step 1: Set up the project
We start with an empty folder and install the packages. AWS CDK describes infrastructure as TypeScript code and generates a CloudFormation template from it. I tested with aws-cdk-lib 2.276, AWS SDK 3.1167, TypeScript 7.0 and Vitest 5.1.
mkdir audiobook-upload && cd hoerbuch-upload npm init -y npm install @aws-sdk/client-s3 @aws-sdk/s3-request-presigner @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb npm install -D aws-cdk aws-cdk-lib constructs esbuild tsx typescript vitest @types/node @types/aws-lambda mkdir bin lib src test web { "compilerOptions": { "target": "ES2022", "module": "NodeNext", "moduleResolution": "NodeNext", "lib": ["ES2022", "DOM"], "types": ["node"], "strict": true, "esModuleInterop": true, "skipLibCheck": true, "noEmit": true }, "include": ["bin", "lib", "src", "test", "web", "vitest.config.ts"] } { "app": "npx tsx bin/app.ts" } In package.json, also add "type": "module" and the scripts "test": "vitest run" and "build": "tsc --noEmit". The full file is in the example repository.
Step 2: Plan parts and validate input
The logic without AWS calls goes into its own file. That way you can cover it with simple unit tests. partLength calculates the length of a single part: all parts have the same size, only the last one takes the rest.
// src/plan.ts // Pure logic without AWS calls: validate input and plan parts. // This makes it testable without mocks. export const MiB = 1024 * 1024; export const MIN_PART_SIZE = 5 * MiB; // S3 minimum for every part except the last export const MAX_PARTS = 10_000; // S3 maximum per multipart upload export const DEFAULT_PART_SIZE = 16 * MiB; // Limit of our example app (audiobook archive), not of S3 export const MAX_FILE_SIZE = 20 * 1024 * MiB; // 20 GiB const ALLOWED_TYPES = new Set([ "audio/mpeg", "audio/mp4", "audio/x-m4b", "application/zip", ]); export interface UploadRequest { fileName: string; contentType: string; size: number; } export interface UploadPlan { partSize: number; partCount: number; } export class ValidationError extends Error {} export function validateRequest(input: unknown): UploadRequest { const body = (input ?? {}) as Partial<UploadRequest>; if (typeof body.fileName !== "string" || body.fileName.trim() === "") { throw new ValidationError("fileName is missing"); } if (typeof body.contentType !== "string" || !ALLOWED_TYPES.has(body.contentType)) { throw new ValidationError("contentType not allowed"); } if (!Number.isInteger(body.size) || (body.size as number) <= 0) { throw new ValidationError("size must be a positive integer"); } if ((body.size as number) > MAX_FILE_SIZE) { throw new ValidationError("File is too large"); } return { fileName: body.fileName, contentType: body.contentType, size: body.size as number }; } // Picks the part size so that there are at most MAX_PARTS parts. // Rounds up to whole MiB to keep the math in the client simple. export function planParts(size: number, preferred = DEFAULT_PART_SIZE): UploadPlan { const needed = Math.ceil(size / MAX_PARTS); const partSize = Math.max(preferred, MIN_PART_SIZE, Math.ceil(needed / MiB) * MiB); return { partSize, partCount: Math.max(1, Math.ceil(size / partSize)) }; } // "My Audiobook (Part 1).m4b" becomes "my-audiobook-part-1-.m4b" – only safe characters in the key export function safeFileName(name: string): string { const cleaned = name .toLowerCase() .replace(/[^a-z0-9._-]+/g, "-") .replace(/^-+/, "") .slice(0, 100); return cleaned === "" ? "file" : cleaned; } export function assertPartNumbers(input: unknown, partCountLimit = MAX_PARTS): number[] { if (!Array.isArray(input) || input.length === 0 || input.length > 100) { throw new ValidationError("partNumbers: 1 to 100 entries"); } for (const n of input) { if (!Number.isInteger(n) || n < 1 || n > partCountLimit) { throw new ValidationError(`invalid part number: ${String(n)}`); } } return input as number[]; } // Length of a part: all parts have partSize, only the last one is shorter export function partLength(plan: UploadPlan & { size: number }, partNumber: number): number { if (partNumber < plan.partCount) return plan.partSize; return plan.size - plan.partSize * (plan.partCount - 1); } Step 3: Store the upload state in DynamoDB
Each upload gets a record with its own ID. The client never sees the S3 upload ID or the object key; it only knows our UUID. Through expiresAt, DynamoDB TTL cleans up old items after one day. TTL stands for time to live: DynamoDB deletes items automatically once the timestamp in that field is reached. The close method uses a condition so that an upload can’t be completed twice or aborted after completion.
// src/store.ts // Small abstraction over DynamoDB so the handlers stay testable without AWS. import { DynamoDBClient } from "@aws-sdk/client-dynamodb"; import { DynamoDBDocumentClient, GetCommand, PutCommand, UpdateCommand } from "@aws-sdk/lib-dynamodb"; export type UploadStatus = "OPEN" | "COMPLETED" | "ABORTED"; export interface UploadRecord { id: string; s3UploadId: string; key: string; contentType: string; size: number; partSize: number; partCount: number; status: UploadStatus; expiresAt: number; // Unix seconds, DynamoDB TTL removes old items } export interface UploadStore { put(record: UploadRecord): Promise<void>; get(id: string): Promise<UploadRecord | undefined>; // Sets the status only while the upload is still open; returns false otherwise close(id: string, status: Exclude<UploadStatus, "OPEN">): Promise<boolean>; } export function dynamoStore(tableName: string, client = new DynamoDBClient({})): UploadStore { const doc = DynamoDBDocumentClient.from(client); return { async put(record) { await doc.send( new PutCommand({ TableName: tableName, Item: record, ConditionExpression: "attribute_not_exists(id)", }), ); }, async get(id) { const res = await doc.send(new GetCommand({ TableName: tableName, Key: { id } })); return res.Item as UploadRecord | undefined; }, async close(id, status) { try { await doc.send( new UpdateCommand({ TableName: tableName, Key: { id }, UpdateExpression: "SET #s = :new", ConditionExpression: "#s = :open", ExpressionAttributeNames: { "#s": "status" }, ExpressionAttributeValues: { ":new": status, ":open": "OPEN" }, }), ); return true; } catch (err) { if ((err as Error).name === "ConditionalCheckFailedException") return false; throw err; } }, }; } Step 4: Write the Lambda handlers
The handlers are created in a factory function that receives the S3 client and the store as parameters. That lets you feed them fakes in tests, without a mocking framework. Pay attention to two places while reading: ContentLength when signing, and the loop with ListParts on completion. That loop is needed because S3 returns at most 1,000 parts per response.
// src/handlers.ts // Four endpoints for the multipart upload: start, sign parts, complete, abort. import { randomUUID } from "node:crypto"; import { AbortMultipartUploadCommand, CompleteMultipartUploadCommand, CreateMultipartUploadCommand, ListPartsCommand, type Part, S3Client, UploadPartCommand, } from "@aws-sdk/client-s3"; import { getSignedUrl } from "@aws-sdk/s3-request-presigner"; import type { APIGatewayProxyEventV2, APIGatewayProxyStructuredResultV2 } from "aws-lambda"; import { assertPartNumbers, partLength, planParts, safeFileName, ValidationError, validateRequest, } from "./plan.js"; import type { UploadRecord, UploadStore } from "./store.js"; type Result = APIGatewayProxyStructuredResultV2; type Handler = (event: APIGatewayProxyEventV2) => Promise<Result>; const URL_TTL_SECONDS = 15 * 60; // each part URL is valid for 15 minutes const RECORD_TTL_SECONDS = 24 * 60 * 60; // matches the lifecycle rule on the bucket const json = (statusCode: number, body: unknown): Result => ({ statusCode, headers: { "content-type": "application/json" }, body: JSON.stringify(body), }); function parseBody(event: APIGatewayProxyEventV2): unknown { if (!event.body) return {}; const raw = event.isBase64Encoded ? Buffer.from(event.body, "base64").toString("utf8") : event.body; try { return JSON.parse(raw); } catch { throw new ValidationError("Body is not valid JSON"); } } // Catches validation errors and turns them into a 400 const safe = (fn: Handler): Handler => async (event) => { try { return await fn(event); } catch (err) { if (err instanceof ValidationError) return json(400, { message: err.message }); console.error(err); return json(500, { message: "Internal error" }); } }; export interface Deps { s3: S3Client; store: UploadStore; bucket: string; now?: () => number; } export function createHandlers({ s3, store, bucket, now = Date.now }: Deps) { // Returns either the open upload or a ready-made error response async function loadOpen( event: APIGatewayProxyEventV2, ): Promise<{ record: UploadRecord } | { error: Result }> { const id = event.pathParameters?.id ?? ""; const record = await store.get(id); if (!record) return { error: json(404, { message: "Upload not found" }Source：https://dev.to/sascha-ms/s3-multipart-upload-with-presigned-urls-and-aws-cdk-4dglRelated posts
Thirty plugins and one of them is abandoned: a plugin audit you can actually finish
One Jira Ticket Through an Org of AI Agents: The Architecture
Find Duplicate and Near-Duplicate Pages Across Your Sitemap with Python (Standard Library Only): Content Hashing, difflib Similarity, CSV Report
Cramming a New Framework Feels Faster. Your Brain Disagrees.
Appointment reminders in a Telegram bot without a scheduler library
