Short answer: give each generated image an immutable object key, record that key in a database at write time, and use a scheduled retention policy plus a storage lifecycle rule to remove old data. The visible "per-user folder" is a key prefix, not a database index or a deletion receipt.
This matters most when an AI feature moves from a notebook into an Express API. A single request can create several candidates; an evaluation run can create many more. If the gallery is built by listing a bucket prefix and retention is judged from a successful job log, the system has no durable account of which image belongs to which run, user, or policy decision.
I would treat the experiment as failed until it can answer two questions from independent data: which objects should exist now, and which objects were actually deleted. Fast uploads are not enough.
What does a per-user folder mean in object storage?
Object storage does not require a real directory for each user. A name such as images/u_8f12/run_20260807/a31c.webp is an object key; the slashes make a useful prefix for people and tools, but the application should not depend on directory semantics. In particular, a prefix is a poor substitute for a gallery query: listing is paginated and ordered by key, while a gallery needs filters, stable pagination, and a record of soft-deleted images.
Use opaque internal identifiers in the key. Email addresses and display names change, and putting them in names can spread personal data into logs and operational tooling. A run identifier earns its place too. It lets a model evaluation retain the exact image set used for a comparison without confusing it with a user's normal gallery.
Here is the small data model I want before adding cleanup code:
| Field | Why it exists |
|---|---|
user_id | scopes authorization and per-user retention |
run_id | groups candidates from one request or evaluation |
object_key | the immutable name used for reads and deletes |
created_at and deleted_at | makes age and deletion state queryable |
byte_size and content_type | supports quotas, cost review, and safe responses |
Keep the delivery URL out of this table as the source of truth. A URL can change with a CDN hostname, a signing policy, or a route rewrite. The exact key is the value that the storage API operates on. Small distinction. Expensive consequence.
How should Express store AI-generated images in per-user object storage and delete old images?
The HTTP handler should validate the authenticated user, construct the key on the server, write the bytes, and persist the exact key and metadata. In a production system, make the database write retryable and reconcile incomplete pairs later: an object without a row is an orphan, and a live row without an object is a missing asset. Neither state should silently become a successful gallery entry.
The following Python is deliberately provider-neutral. It shows the boundary an Express route or a Python worker needs: the route passes bytes and trusted IDs into an application service; the application service owns key construction and writes one ledger row. It does not construct a key from a client-supplied filename.
from dataclasses import dataclass from datetime import datetime, timezone from hashlib import sha256 @dataclass(frozen=True) class GeneratedImage: user_id: str run_id: str object_key: str byte_size: int created_at: datetime def make_object_key(user_id: str, run_id: str, payload: bytes) -> str: digest = sha256(payload).hexdigest() return f"images/{user_id}/{run_id}/{digest}.webp" def save_generated_image(store, images, user_id: str, run_id: str, payload: bytes) -> GeneratedImage: key = make_object_key(user_id, run_id, payload) store.put(key=key, body=payload, content_type="image/webp") image = GeneratedImage( user_id=user_id, run_id=run_id, object_key=key, byte_size=len(payload), created_at=datetime.now(timezone.utc), ) images.insert(image) return image For large uploads, use a multipart-upload protocol rather than treating a whole file as one request. Multipart upload divides an object into parts that can be uploaded independently, then completes as one object; incomplete multipart uploads need their own cleanup policy. That detail is easy to miss in an image pipeline because individual generations start small, then a higher-resolution model or a batch export changes the input size without changing the handler.
Secrets belong outside source code, notebooks, prompts, and request logs. Inject short-lived or scoped credentials through the deployment environment or a dedicated secret-management system, and make storage permissions narrow enough that one user-facing process cannot enumerate unrelated prefixes. OWASP's guidance is useful here because a storage credential has a much larger blast radius than an image-processing failure.
The retention policy needs two layers
Use the database for rules that understand product state. "Keep the newest 30 images for this user," "retain evaluation runs for 14 days," and "hold images attached to an active review" all require ordering and metadata that an object name alone cannot provide. A scheduled worker can select the rows beyond the rule, attempt deletion using their stored object_key, and mark only the confirmed outcomes according to the API's documented response semantics.
Then add a lifecycle rule in the storage system as a coarse backstop for the images/ prefix. Lifecycle policies are good at age-based expiry and at aborting incomplete multipart uploads. They are not suitable when the business rule depends on the newest N objects per user, a legal hold, or a review state; keep those decisions in the ledger.
The catch is that lifecycle expiry is usually an asynchronous storage operation, so it is not a deletion acknowledgement suitable for a user-facing "erase now" promise. For a deletion requirement that must be demonstrable per object, keep an explicit audit trail and confirm the storage-side outcome. For a low-risk thumbnail cache, an age-based lifecycle policy may be the entire answer. Your retention window should be driven by reproducibility and product expectations, not by an assumed storage bargain.
There is a useful ordering to these decisions. First decide what the user is allowed to see and for how long; that is an application policy, so it belongs in a queryable record. Next decide how a background worker identifies candidates without racing a fresh upload: a row state, an eligibility timestamp, and an idempotent attempt record make that decision inspectable. Then decide how long untracked or abandoned bytes may remain before lifecycle expiry clears them. The three layers serve different failure modes, which is why compressing them into one prefix convention is tempting but wrong. A prefix can scope a broad rule. It cannot explain why one image was retained while its neighbor was erased, nor can it reconstruct the image set that produced an evaluation result after the fact. Those answers come from application metadata.
Measure the cleanup path before trusting it
Start with a small reconciliation job. Select a sample of active users, compare ledger keys with keys observed under each prefix, and publish counts for orphaned objects, missing objects, bytes by retention class, and the age of the oldest eligible object. An evaluation-driven image workflow benefits from one more metric: how long an evaluation artifact remains reproducible after the test result was recorded.
One short run is enough to reveal a lot.
def reconcile_prefix(images, store, user_id: str) -> dict[str, int]: expected = set(images.active_keys_for_user(user_id)) observed = set(store.list_keys(prefix=f"images/{user_id}/")) return { "orphan_count": len(observed - expected), "missing_count": len(expected - observed), "expected_count": len(expected), "observed_count": len(observed), } Don't alert only on worker exceptions. A worker can run on schedule while deleting the wrong keys, while a lifecycle rule can remove objects the ledger still considers live, or while a failed database transaction leaves a newly uploaded object untracked. The reconciliation numbers expose those mismatches directly. Tune the sample size, retention windows, and retry behavior after observing real request volume; the correct threshold depends on image size, generation rate, and the cost of losing a reproducible evaluation artifact.
The practical design is modest: immutable keys, a ledger, explicit policy, and a backstop. It leaves room for an Express API today and batch-oriented Python evaluation work tomorrow without making bucket listings carry product semantics they do not have.
