Early on 22 November (UTC) an automated check caught the "Try it free" photo button of AllKeep Inventory, a home inventory web app, answering every try that reached it with a 500 (02:50 to 04:14 UTC), because the AI account behind it had run out of credit. Our error tracker had already counted 39 such errors in the week before. Topping the account up would have fixed one of four things wrong with that page. The other three were waiting behind it. A failed call spent the visitor's only free try, a 7.6 MB phone photo was refused before our code ran, and once the AI answered again, one of three answers came back with the category "null".

The page

AllKeep Inventory keeps a list of what you own, room by room and box by box, with photos and values, so there is something to hand an insurer after a fire or a burglary. Its landing page has a "Try it free" button: you take one photo of anything, with no account, and an AI model names the thing, writes a line about it and picks a category. One try per device; after that the page asks you to sign up.

AllKeep is an experiment. AI coding agents write, test and review it, and I direct them and check what they measure. The fixes below all landed on 22 November, and the numbers come from their logs, test runs and pull requests.

The flow is short. The page reads the photo into a base64 data URL and POSTs it as JSON to a Next.js route on Vercel. The route checks that this device has not used its try, calls a vision model, and returns a name, a description, a category and tags.

1. The 500, and what the visitor read

An automated check of the live page saw the try fail at 02:50 UTC on a plain photo of a coffee mug. At 04:13 and 04:14 another photo, 155 KB, got a 500 twice, and the runtime log on Vercel said why:

OpenAI API error: You have no credits remaining. Add credits to continue using the API 

The account is prepaid, and that morning every AI call in the product went through it. Our error tracker already knew. Its weekly email, sent the day before, listed this error as the top issue of the week with 39 events.

What a visitor saw was worse than the 500. The page had two failure screens. A 429 got "That's your free try.", and anything else got a screen that told the visitor to try a different photo. So a person who came to see what the app does was told their photo was the problem.

2. A failed call spent the only free try

The route takes the try first and calls the model second, so no paid call runs without a try on record. But nothing gave the try back when the call failed. The row in the anon_attempt table stayed at attemptCount: 1, and the visitor who met the outage got "That's your free try" on their next photo.

The fix releases the reservation when the model call throws, and only then:

let result; try { const provider = createRoutedAIProvider(); result = await provider.analyzeImage({ imageUrl: imageBase64, language, analysisType: "full", // No workspace yet, so empty lists let the provider fill in its defaults. userContext: { categories: [], tags: [], currency: "USD" }, }); } catch (error) { await prisma.anonAttempt.updateMany({ where: { deviceId, attemptCount: 1 }, data: { attemptCount: 0 }, }); throw error; } 

An AI failure now answers 503 with the code AI_SERVICE_ERROR instead of a 500, and the page has a third screen for any 5xx: "Our AI can't read photos right now", then "your photo is fine", with buttons to create a free account or try again. The OpenAI text goes to the log and the error tracker only.

One smaller thing came out of the code review. After a good answer the route saves the label on the same row, so it can create the item without a second model call once the visitor signs up. If that write failed, the whole request answered 500, which threw away a result the visitor had already paid for with their try. That write is now best effort: logged, and the 200 goes out anyway.

3. A phone photo bigger than the request limit

Even with credit, a big photo straight from a phone camera would still have failed. A 7.6 MB phone photo got this from Vercel before the route ran at all:

413 FUNCTION_PAYLOAD_TOO_LARGE 

Vercel's functions take a request body of at most 4.5 MB by default, with a 20 MiB limit as an opt-in beta. The page sent the original file as a base64 data URL inside JSON, and base64 writes every 3 bytes as 4 characters. So the 7.6 MB file became a body of about 10.1 MB, and any file over about 3.4 MB was going to be refused, whatever the model's account held. None of the 11 AI fixture photos in the repo is over 330 KB, so nothing in our tests came near the limit.

The 7.6 MB photo sent as base64 is about 10.1 MB, more than twice Vercel's 4.5 MB request limit; downsized to a 1600 px JPEG it is a 0.28 MB request

The page could have opted in to the bigger limit. Shrinking the photo in the browser instead also cuts what a phone uploads, from about 10.1 MB to 0.28 MB, so the fix makes the browser do the shrinking. The page draws the photo onto a canvas at most 1600 px on its long side and sends that as a JPEG:

export const MAX_SIDE = 1600; export const JPEG_QUALITY = 0.85; export async function photoForAnalysis(file: File): Promise<string> { try { const bitmap = await createImageBitmap(file, { imageOrientation: "from-image" }); try { const { width, height } = fitWithin(bitmap.width, bitmap.height); const canvas = document.createElement("canvas"); canvas.width = width; canvas.height = height; const ctx = canvas.getContext("2d"); if (!ctx) throw new Error("No 2D canvas"); ctx.fillStyle = "#fff"; ctx.fillRect(0, 0, width, height); ctx.drawImage(bitmap, 0, 0, width, height); return canvas.toDataURL("image/jpeg", JPEG_QUALITY); } finally { bitmap.close(); } } catch { return fileToDataUrl(file); } } 

In headless Chrome the 7.6 MB photo went out as 1200x1600 in a 278 KB request. The 155 KB photo, already small at 768x1024, went out at its own size in a 175 KB request.

imageOrientation: "from-image" applies the photo's EXIF rotation, so a portrait photo is drawn upright. The white fill came from the code review: the first version turned the transparent parts of PNG and WebP files black. A file the browser cannot draw is sent as it is.

The server got its own limit too: the route refuses a base64 string over 4 MiB, a little under Vercel's 4.5 MB.

One case is left. A HEIC file over about 3 MB, in a browser that can't draw HEIC, still goes as it is and still fails on size. The screen it gets says "try a different photo", which for that case is the right advice.

4. When it worked, the category said "null"

At 21:35 UTC, from a fresh IP, the coffee-mug photo answered 200 in 3.3 seconds (the page tells the visitor to expect about 8 to 14 seconds): "Ceramic Coffee Mug", a description, the category "Kitchen & Dining". A second try from the same device answered 429, as it should. A forced 503 on the live page showed the new "your photo is fine" screen.

In the same check, one of three calls came back with "category": "null", and the page printed it: CATEGORY null.

The prompt has placeholders for the user's own categories, tags and currency. This is where they are filled in:

if (request.userContext?.categories) { const categoriesList = request.userContext.categories.length > 0 ? formatCategoriesForPrompt(request.userContext.categories) : getDefaultCategoriesString(); promptText = promptText.replace("{CATEGORIES}", categoriesList); } 

The fallback to the default list is there, but it sits inside the if. A signed-in user always has a context, so their prompt was always filled in. The anonymous route has no user and passed no context at all, so the whole block was skipped, and the same happened to {TAGS} and {CURRENCY}. The model was asked to pick a category from the literal text {CATEGORIES}, and sometimes it answered with the word "null".

The fix is two lines and a filter. The route passes an empty context (empty lists, USD), which sends the provider down the default-list path it already had; that is the userContext line in the snippet in section 2. And a category that comes back empty or as "null", "none", "undefined" or "n/a" is dropped, so the page hides the row instead of printing the word. Seven of the 24 tests in the route and provider suites failed against the main branch; with the change, those 24 and the router's four, 28 in all, pass. The live check that posts the mug photo to production now also fails when the description or a real category is missing.

The hole the outage made us look at

While reading the route we looked at what "one try per device" means. The device is a UUID that the page makes itself and sends in a header. A new UUID is a new device, so anyone with a loop could make a paid vision call per request until the prepaid account was empty, the same state that caused the outage.

The route now holds three limits before it calls the model:

  • one IP address gets at most 3 tries a day, read from x-real-ip. Vercel sets that header itself and overwrites any forwarded address a client sends, "to prevent IP spoofing", so a client cannot forge it there;
  • all anonymous tries together stop at a fixed number a day, set in ANON_AI_DAILY_CAP; over that the route answers 503 and the page shows the "can't read photos right now" screen;
  • ANON_AI_OFF=1 turns the route off with a redeploy and no code change.

Three new tests fail against the old route and pass now.

A second provider behind the first

The outage also showed that one empty account stops the product's first step. Later that day the agents added a Gemini provider beside OpenAI, reusing the same prompts and the same parsing of the answer, and a small router that tries the provider named in AI_PROVIDER and hands the same request to the other one when the first fails on its own side.

This is where the router decides what counts as "its own side":

export function switchable(error: unknown): boolean { if (error instanceof AppError) { // the provider's own trouble; a validation error is the request's and goes back to the caller return ( error.code === "AI_SERVICE_ERROR" || (error.statusCode ?? 0) >= 500 || error.statusCode === 429 || error.statusCode === 402 ); } return true; // a network error, a timeout, an unexpected response } 

Quota and credit answers from either side (HTTP 429 and 402, Gemini's RESOURCE_EXHAUSTED, OpenAI's insufficient_quota and "no credits remaining") are marked as one error type and never retried on the same provider. A bad request goes back to the caller, because the other provider would refuse it too. Unit tests mock both providers' HTTP, including a bad-key 400 that must not count as quota.

The first model the agents wired in was gemini-2.5-flash, which Google had already deprecated: it is gone from AI Studio, and from 1 December 2026 its API requests go to another model (Gemini deprecations). Our new key could not call it at all; the API answered that the model is "no longer available to new users". The key worked on gemini-3-flash-preview, a preview model that the same page already lists with gemini-3.6-flash as its replacement. So the model name now comes from GEMINI_MODEL, and the next swap needs no code change, only a redeploy.

The check at 21:35 UTC in section 4 ran after the router was merged and did not record which provider answered, so I can't say whether the OpenAI account had credit again by then.

What I still don't know

I have no count yet of how often the model returns no category after the merge, against one of three "null" answers before it.

I also don't know how best to limit a free AI call for people with no account. Our device id is made by the client, so it limits nothing on its own. The IP limit is the only one the client does not make up itself, and it is crude, because a mobile network or an office can put many people behind one address, so the fourth visitor of the day from there gets "That's your free try." without having tried. The daily cap only limits the total.

You can try the photo yourself on the AllKeep Inventory page with the "Try it free" button.

If you run a free, no-sign-up AI call on a public page, what do you limit it by (a captcha before the call, a signed token from the page, proof of work, a cap per IP, something else), and how many ordinary visitors did it turn away when you changed it?