How ScreenWeaver finds the panels on a comic, webtoon or storyboard page with Gemini Flash, and the small decisions that keep it cheap and safe to run twice.

A comic page is already half a storyboard. Six frames, a reading order, a page number at the bottom. If you want to turn that page into an animated scene, every panel has to become its own image so it can sit on a shot as a reference. Cutting them by hand in an image editor is easy for one page. Do it for a whole chapter and you've spent hours on crop boxes before making a single shot.

Comic panel detection with a vision LLM means sending one page image to a multimodal model, asking it for a bounding box around every panel in reading order, then cutting those boxes on your own server. In ScreenWeaver, where I'm a co-founder, it runs on Gemini Flash and costs between $0.0036 and $0.0048 a page in our measurements.

We shipped it on October 13, 2026. This is a look at how it works, and at the dull decisions around the model call that ended up mattering more than the model itself.

Why not just use OpenCV contour detection?

Classic panel detection works, and it's free. The best-known open-source option is Kumiko, the Comics Cutter, which uses OpenCV's contour detection to find panels and can number them left to right or right to left. It runs locally and costs nothing per page.

So why pay for a model call? Because of the pages we actually get:

  • Storyboard sheets with notes. Someone photographs a notebook page: six pencil sketches, a few words of French under each. Contours will happily pick up the handwriting.
  • Webtoon strips. Long vertical images with wide colored gutters, and sometimes panels with no border at all.
  • Panels that bleed. A splash image running to the edge of the page, a sound effect breaking out of its frame.

And we wanted one more thing contours can't give you: a one-line caption per panel, saying who is in it and what happens. That caption travels with the cropped image, so the agent that builds the shots later doesn't have to look at every panel again to know what it is.

A contour finder sees rectangles. A notebook page full of handwriting has plenty of rectangles that aren't panels.

To be fair to Kumiko: if your pages are clean, with clear gutters around every frame, it's free, offline and worth trying first.

Kumiko, the open-source comics cutter, on GitHub. Screenshot: github.com/njean42/kumiko, October 2026.

What does the prompt actually ask the model for?

One page per call. The page is turned upright, flattened on white, shrunk so its longest side is 1,568 pixels and sent as a JPEG. The system prompt is short. It asks for every panel in reading order, each with a box and a caption of 80 characters at most, and then it spends most of its words on what to leave out: the gutters, page numbers, titles, and notes written beside or below a frame on a storyboard sheet. A speech balloon inside a frame stays part of the panel.

Two edge cases are spelled out because they come up constantly. If the whole page is one picture, the model returns one box around it. If the page has no picture at all (a text page, a blank page), it returns nothing.

The box format comes straight from Google. Gemini is trained to return boxes as [ymin, xmin, ymax, xmax] on a 0 to 1000 scale relative to the image, and the Gemini API documentation tells you to scale those values back to your original image size yourself. We use structured output, so the response is a JSON object we validate with a schema rather than text we parse with regular expressions. Reasoning is switched off for this call: it's a perception task, and thinking tokens would only add cost.

The object detection section of Google's Gemini API docs, where the 0 to 1000 box scale is defined. Screenshot: ai.google.dev, October 2026.

How do you clean up what the model returns?

You don't trust it as is. Model output is input from outside your code, and one malformed box is enough to break a crop. Before anything gets cut, every box goes through the same small filter:

  • Clamp every coordinate to the page, so a box can't run past the edge.
  • Fix swapped edges: if xmin comes back bigger than xmax, take the smaller one as the left edge instead of producing a negative width.
  • Drop specks. Any box smaller than 1% of the page is thrown away. It's almost always a page number or a stray mark.
  • Cap the count at 40 panels per page, in reading order.
  • Trim the caption to 80 characters.

Then the server cuts each box out of the full-resolution page (the model only saw the 1,568 px version, but the coordinates are fractions, so they apply to the original) and stores every panel as its own image, labelled "Page N, panel K" with its caption.

None of this is clever. The filter is about twenty lines of plain code, and it's the reason a bad box costs you one missing panel instead of a failed import.

Where the model call sits in ScreenWeaver's panel detection. Diagram: ScreenWeaver.

What does it cost, and who decides to spend it?

The measured cost sits between $0.0036 and $0.0048 a page. One real run: a 20-page storyboard with six panels a page came to $0.097 in total. For budgeting, the app counts about $0.005 a page, which is one credit.

Because this is the step of an import that spends money, it has guard rails:

  • The balance is checked first. If the wallet can't cover about a credit a page, nothing is sent to the model.
  • Above 20 pages, the call stops and asks. The import tool answers confirmation_required until it's called again with confirm: true. An AI agent working through the ScreenWeaver MCP server can't quietly push a 300-page volume through detection.
  • There is a free path. An agent can pass its own boxes to the same tool (crops) and the server just cuts the pixels. No model call, no credits. If your agent can already see the page and draw boxes, detection is optional.

The model call is the cheap part. The expensive part is running it twice by accident.

What happens when you run the import again?

This is the decision I'd copy into any pipeline that calls a paid model on files.

The panels are saved on the page record itself. If an import is retried, after a timeout or because the agent decided to add more crops, the stored panels are reused. The model isn't called again, nobody pays again, and the boxes don't move. That last part matters more than the money: if a second run nudged every box by a few pixels, every shot that already uses a panel would quietly change.

The IDs make this work. A page's ID is built from a hash of the file plus the page number, and a crop's ID from the page ID plus a hash of the box. Import the same PDF next week and the server recognizes every page and every panel it already rendered, and renders nothing twice.

Failures stay local too. If the model call fails on page 14, that page is reported as panelsFailed and the import carries on with page 15. You get panels for every other page and a clear note about the one that needs another try, instead of an error and nothing.

What a first import costs and what a re-import costs. Figures from ScreenWeaver's own measurements, October 2026.

Does it work on webtoons and manga?

Webtoons, yes, with one preparation step. A webtoon episode is often a single strip thousands of pixels tall. Shrunk to 1,568 px on its longest side, the panels would be too small to box. So a long strip goes in as a series of images about the height of a page. We did this on Imaginarium, a webtoon adapted into an animated series in ScreenWeaver: pages of episode 1 went in, the panels came out, and each one became the reference for an animated shot.

Manga, with a caveat I'd rather say out loud. The prompt asks for "reading order" and doesn't say which direction. We haven't measured the result on a run of right-to-left pages yet, so I won't claim the order comes out right every time. If you're processing manga in volume today, check the order on a few pages first. This is one place where Kumiko's explicit right-to-left switch is simply better.

Storyboard sheets are where the vision model earns its money. On that notebook page of six pencil sketches, the notes under each frame ("plan large", "travelling avant", "insert fleur") stay out of the cropped images. The agent read them separately and used them to set the framing and camera move of shots S14 to S19, with each sketch attached to its shot.

A real notebook page imported through the ScreenWeaver MCP, each sketch stored as its own image. Diagram: ScreenWeaver.

What happens after the panels are cut?

Panels on their own are just images. The rest of the chain is what turns a page into a scene:

  • the agent lists who and what keeps coming back across the panels, asks the author once, and creates those characters and places as story elements (create_entities);
  • it writes the shots (create_shots), each citing the elements it shows;
  • it attaches the panel to its shot as a reference image (add_reference_images) and writes the image prompt (set_prompt), with the speech balloons left out of the shot.

Then a person opens the Storyboard and Production tabs and launches the generations. The MCP server prepares; it never generates images or video on its own. That boundary is deliberate: the agent does the tedious part, the author keeps the button that spends real money on footage.

If you want to see it on a real webtoon, the Imaginarium pages and the shots they became are on our webtoon page.

FAQ

Which model does ScreenWeaver use for panel detection?

Gemini Flash (google/gemini-3.8-flash) through an AI gateway, with structured output and reasoning switched off.

Can I detect comic panels without paying for an API?

Yes. Kumiko does it locally with OpenCV contour detection. It suits clean pages with clear gutters best.

How many panels per page can it return?

Up to 40, in reading order. Boxes under 1% of the page are dropped.

Does it remove speech balloons?

Detection keeps balloons inside the panel. They're left out later, when the panel is used as a reference for a shot.

If you're building something similar, spend your time on the code after the model call, and store what the model told you so you never pay for the same page twice. The panel boxes were the easy part.