Your width filter does not certify downloaded pixels

Your width filter can pass a photo that arrives at thumbnail size.

You're checking why the training job rejects images your ingestion job already approved. Another evening disappears into a filter that looks perfectly reasonable.

minWidth and minHeight filter original media dimensions, while imageSize selects the rendition served through photoUrl.

Example inputs keep the same source thresholds while changing the requested rendition:

const originalInput = { mode: "place", placeUrls: ["Empire State Building"], photoCategory: "photos", minWidth: 1200, minHeight: 1200, imageSize: "original", }; const run_input = { ...originalInput, imageSize: "thumbnail", };

Both inputs are valid. Neither turns source metadata into a measurement of downloaded bytes.

The trap is treating adjacent settings as a shared guarantee: your selection rule approves the original, but your training loader receives a particular representation of it.

Why can a width filter admit a thumbnail?

The gallery Actor separates media selection from URL rendition selection because those decisions serve different stages of a workflow.

You might need a large original eventually, while a small preview is enough to decide whether the photograph contains anything useful. Downloading everything at maximum size makes that decision unnecessarily expensive.

The thumbnail rendition is about 200 px. Selecting original requests the largest version the host serves.

Neither setting changes the meaning of the source dimensions. They describe the original media, even when you request a smaller representation.

That distinction belongs in your data model:

Evidence	What it answers	What it cannot establish
Source `width` and `height`	Does the original have sufficient dimensions?	The geometry of fetched bytes
Requested `imageSize`	Which rendition did you request?	Successful delivery or decoding
Dimensions inspected from the file	Does this representation meet your size rule?	Visual detail, relevance, or training suitability

A column named width becomes expensive when your ingestion job, downloader, and training loader quietly assign it different meanings.

Give each measurement a subject. Your future debugging session deserves nouns.

Unknown dimensions are still candidates

The Actor keeps media whose dimensions are unknown, and it omits fields the source doesn't publish.

That preserves potentially useful candidates. Missing evidence of size doesn't establish that an image is too small, so excluding it would make metadata availability decide which images you can examine.

Your consumer needs a stricter boundary.

Passing the source filter means the record remains eligible for inspection; it doesn't mean the delivered image has earned a place in your training corpus.

This workflow puts that boundary after delivery:

public media | source filters | candidate + selected rendition | bounded download | file geometry check | full decode and training checks

Known dimensions help you avoid unnecessary downloads before transfer. Unknown dimensions reach the same file check, without an exemption or an invented measurement.

The file check can save expensive downstream work. It can't refund the bytes that reached it.

Keep source metadata separate from delivery evidence

Validate the fields your consumer needs without converting missing dimensions into zeros.

That distinction matters when you investigate rejections: an unavailable source measurement and a measured undersized original explain different things, even if neither provides source qualification.

This cell retrieves a dataset page and defines the record shape used below:

import { ApifyClient } from "apify-client"; import { z } from "zod"; import { Buffer } from "node:buffer"; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client .actor("nimbryxo-studio/google-maps-photo-gallery-extractor-by-category") .call(run_input); if (run.status !== "SUCCEEDED") { throw new Error(`Gallery run ended with ${run.status}`); } const { items } = await client.dataset(run.defaultDatasetId).listItems(); const PhotoRow = z.object({ photoId: z.string().min(1), photoUrl: z.string().url(), width: z.number().int().positive().optional(), height: z.number().int().positive().optional(), mediaType: z.enum(["photo", "video", "streetView"]).optional(), isVideo: z.boolean().optional(), isStreetView: z.boolean().optional(), });

Requiring an identifier and usable URL is your admission policy. A record that can't satisfy it goes to quarantine, where you can inspect the failure instead of silently repairing the evidence.

Keep media classification explicit too. recordType remains "photo" for backward compatibility, including video rows, and a video's main image URL points to its poster frame.

A poster can pass a dimension gate perfectly. That doesn't make it the kind of photograph you intended to collect.

The consumer below requires mediaType to identify a still photo and also checks the video and Street View flags. Missing classification stays outside this queue.

Dataset traversal is a separate concern. This example handles the retrieved page; consuming a page doesn't establish that you've consumed the dataset or exhausted a place's gallery.

Inspect the representation you fetched

Node has no general image decoder built in. With the package set used here, a small header probe can inspect raster dimensions before your existing training loader performs a full decode.

The probe below recognizes PNG and JPEG. Unsupported encodings remain unqualified rather than acquiring dimensions copied from the source record.

Keep the distinction clear: reading dimensions from a header doesn't prove the remaining file is intact, and it doesn't establish how much useful detail the image contains.

This cell bounds each transfer and inspects the delivered header:

type Dimensions = { width: number; height: number }; function probeDimensions(bytes: Buffer): Dimensions | undefined { const dimensions = (width: number, height: number) => width > 0 && height > 0 ? { width, height } : undefined; const png = Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]); if ( bytes.length >= 33 && bytes.subarray(0, 8).equals(png) && bytes.readUInt32BE(8) === 13 && bytes.toString("ascii", 12, 16) === "IHDR" ) { return dimensions(bytes.readUInt32BE(16), bytes.readUInt32BE(20)); } if (bytes.length < 4 || bytes.readUInt16BE(0) !== 0xffd8) return; let offset = 2; while (offset < bytes.length) { if (bytes[offset++] !== 0xff) return; while (offset < bytes.length && bytes[offset] === 0xff) offset++; if (offset >= bytes.length) return; const marker = bytes[offset++]; if ([0x00, 0xda, 0xd9].includes(marker)) return; if (marker === 0x01 || (marker >= 0xd0 && marker <= 0xd8)) continue; if (offset + 2 > bytes.length) return; const length = bytes.readUInt16BE(offset); if (length < 2 || offset + length > bytes.length) return; const isFrame = marker >= 0xc0 && marker <= 0xcf && ![0xc4, 0xc8, 0xcc].includes(marker); if (isFrame) { if (length < 8) return; return dimensions( bytes.readUInt16BE(offset + 5), bytes.readUInt16BE(offset + 3), ); } offset += length; } } const maxDownloadBytes = 16 * 1024 * 1024; const downloadTimeoutMs = 15_000; async function downloadBounded(url: string): Promise<Buffer> { const response = await fetch(url, { signal: AbortSignal.timeout(downloadTimeoutMs), }); if (!response.ok || !response.body) { await response.body?.cancel(); throw new Error(`Image request failed: HTTP ${response.status}`); } const reader = response.body.getReader(); const chunks: Uint8Array[] = []; let total = 0; try { while (true) { const part = await reader.read(); if (part.done) break; total += part.value.byteLength; if (total > maxDownloadBytes) throw new Error("Transfer budget exceeded"); chunks.push(part.value); } return Buffer.concat(chunks, total); } finally { await reader.cancel().catch(() => undefined); reader.releaseLock(); } }

These transfer limits are local consumer choices. Set them for your workload, then keep a separate pixel allocation limit in the decoder, since compressed byte size doesn't bound decoded memory.

If your loader applies orientation transforms or crops before admission, measure that resulting representation there too. The dimensions that matter depend on the stage making the decision.

Most importantly, decode the same bytes you inspected. Fetching the URL again for training would introduce another delivery whose properties you haven't checked.

Give delivery failures their own lane

Public image URLs can change or disappear. A failed request doesn't tell you whether the original had adequate resolution.

Keep request failures, unsupported headers, and undersized deliveries separate, or your rejection metric will combine problems that require different fixes.

For this worked example, Google Maps Photo Gallery Extractor by Category supplies the candidates. The reusable pattern is to retain source claims alongside evidence collected from each delivery.

This cell records both measurements and groups outcomes by requested rendition and source qualification:

const quarantined: unknown[] = []; const skipped: unknown[] = []; const readyForDecode: Array<{ photoId: string; bytes: Buffer }> = []; const counts = new Map<string, number>(); const observations: Array<{ photoId: string; rendition: string; sourceWidth: number | null; sourceHeight: number | null; downloadedWidth: number | null; downloadedHeight: number | null; sourceQualified: boolean; outcome: string; failure: string | null; }> = []; for (const raw of items) { const parsed = PhotoRow.safeParse(raw); if (!parsed.success) { quarantined.push(raw); continue; } const row = parsed.data; if ( row.mediaType !== "photo" || row.isVideo === true || row.isStreetView === true ) { skipped.push(raw); continue; } const sourceQualified = row.width !== undefined && row.height !== undefined && row.width >= run_input.minWidth && row.height >= run_input.minHeight; let measured: Dimensions | undefined; let outcome: string; let failure: string | null = null; try { const bytes = await downloadBounded(row.photoUrl); measured = probeDimensions(bytes); if (!measured) { outcome = "needs-probe"; } else if ( measured.width < run_input.minWidth || measured.height < run_input.minHeight ) { outcome = "delivery-too-small"; } else { outcome = "size-qualified"; readyForDecode.push({ photoId: row.photoId, bytes }); } } catch (error) { outcome = "delivery-error"; failure = error instanceof Error ? error.message : "Download failed"; } observations.push({ photoId: row.photoId, rendition: run_input.imageSize, sourceWidth: row.width ?? null, sourceHeight: row.height ?? null, downloadedWidth: measured?.width ?? null, downloadedHeight: measured?.height ?? null, sourceQualified, outcome, failure, }); const group = JSON.stringify([ run_input.imageSize, sourceQualified, outcome, ]); counts.set(group, (counts.get(group) ?? 0) + 1); }

The observations preserve uncertainty in separate source and download columns. The grouped counts let you isolate deliveries that failed the size gate despite qualifying source metadata, with the requested rendition attached.

A false source qualification means the available metadata didn't establish eligibility. The retained dimensions let you distinguish missing evidence from an explicit measurement below your threshold.

The queue retains bytes for the decoder. For continuous ingestion, hand each accepted buffer to your loader or object storage and release it; a bounded file size doesn't bound total batch memory.

Persist the delivery observation alongside the stored asset. Keeping dimensions while discarding which representation they describe recreates the original bug with better column names.

What broke was the admission rule

The broken assumption is that a source-qualified record represents a training-qualified file.

The Actor permits a small rendition of a sufficiently large original and retains candidates with unknown source dimensions. Both behaviors support useful workflows, but neither supplies file-level approval.

Upscaling an undersized delivery doesn't recover discarded detail. It can satisfy a tensor shape while giving the model enlarged blur.

For inexpensive visual selection, use previews to choose candidates, then request a larger rendition for accepted media. Join later results using photoId, the identifier intended for deduplication, rather than relying on matching positions across retrievals.

That identity still needs a delivery history. If you upsert every observation solely by media identifier, a later preview can overwrite measurements associated with a previously stored larger asset.

We'd keep media identity stable and attach delivery measurements to the particular asset retained by the consumer. A requested rendition helps explain the transfer, but it isn't a content identity.

Also account for the selection bias introduced by previews. Fine text or subtle visual features may disappear at preview size, so rejecting those previews can discard originals your task would have accepted.

A preview workflow saves transfers only when its selection step has enough information to make the decision. For OCR or detail-sensitive training data, that trade-off deserves checking before you optimize download size.