Every practice test on CogniPrep has a card, and every card carries a
screenshot of that test in play. There are 491 tests and 491 images, and the
one-to-one is not luck: the list the cards read from is generated by reading the
directory.

/** * Games with a card screenshot at public/games/screenshots/<id>.webp. * Generated by `pnpm tsx scripts/game-screenshots.ts manifest`; do not edit by hand. */ export const GAME_SCREENSHOTS: ReadonlySet<string> = new Set([ 

That generated set buys one guarantee: a card can never point at a file that is
not there. It buys nothing at all about whether the picture still shows the
thing it is captioned with. An image is the one asset type in a repository that
can go quietly, completely wrong while every test stays green.

Last weekend made that concrete. A round of work rebuilt thirteen tests to match
the real instruments more closely, and some of them look nothing like they did
on Friday. The round added 26 new images and retook 13. Nothing in the repo told
me which 13. I knew because I had just done the work.

So this week I measured it.

The audit

Of the 491 tests, 281 have their own component directory named after the test's
id. For each of those, compare two dates: the last commit touching
components/games/<id>/, and the last commit touching
public/games/screenshots/<id>.webp.

audited 281 image older than its component code 58 largest gap 39 days 

58 of 281. A fifth of the tests I can check have a card picture that predates
the last change to the code that draws them.

The number is an upper bound, and here is the proof

The biggest gap in that list is a test called aon-switch: code touched on 20
November, image captured on 12 October, 39 days apart. Here is the commit that
created the gap:

86c2b364 Point Aon switch comments at the current Saville diagrammatic model components/games/aon-switch/AonSwitchEngine.ts | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) 

Four lines changed, all of them comment text. The image is perfectly accurate.
The audit flagged it because the only thing a date comparison can detect is
"something in here changed", and most commits in a game directory change no
pixels at all: a scoring tweak, a type move, a test fix, a comment pointing at
the right model.

And it is not a ceiling either

The other 210 tests have no directory of their own. They are parameterised reuses
of another test's engine, declared in their provider's component map with a
different item bank and title. There are 71 *-shared directories in the games
folder, and a change in any one of them can change the look of tests in
several providers at once, none of which the audit above examines.

So the honest summary of the measurement is: 58 suspects out of the 281 I can
see, most of which are innocent, drawn from the 57 per cent of the catalogue
where the question is even askable.

One more wrinkle that flatters the result. The image dates cluster hard:

2026-10-12 207 2026-11-12 18 2026-11-20 17 2026-11-19 16 2026-11-16 13 ... 

207 of the 281 images were committed on one day in October, during a mass
recapture. So for most of the catalogue "image date" does not mean "the day
somebody looked at this picture", it means "the day somebody recaptured
everything". The comparison is coarse in both directions at once.

The field that would fix it

What is missing is provenance at capture time. The capture script already knows
everything needed: it renders the test in a browser, crops, encodes and then
regenerates the manifest. The one thing it does not record is what it was looking
at.

The manifest is a Set of ids. Making it a Record<string, { sha, capturedAt }>
costs nothing at call sites that only ask "does this id have an image", and gives
a check that can run in CI: for each id, is the current git log -1 of the files
that render it newer than the sha recorded when the image was taken? That still
produces false positives such as the comment commit above, but it produces them
as a shortlist a human can clear in a minute, instead of as a question nobody
asks.

Pixel comparison, the obvious alternative, does not work here. These previews
render live tests whose content varies between runs, so two captures of an
unchanged test differ. The only machine-readable signal is provenance, and
provenance has to be written at the moment of capture or it does not exist.

See it

The images are public, so you can do the comparison yourself without the repo.

  1. Open /games/test-partnership and look at the card for the inductive reasoning test. Then open the page for that test, /games/test-partnership/inductive-reasoning, which describes the format as six boxes with a gap in one of them. The picture and the prose agree, because that one is among the 13 retaken this weekend.
  2. Fetch a card image directly and you get the raw asset, 960 pixels wide, encoded at quality 80:
 curl -sI https://cogniprep.app/games/screenshots/tp-inductive.webp curl -sI https://cogniprep.app/games/screenshots/aon-switch.webp 

The second one is the 39-day "stale" image from the audit. Open it next to
/games/aon/switchchallenge
and you can confirm by eye what the commit already told me: the date gap was a
comment.

  1. Pick any provider hub, for example /games/saville, and treat each card as a claim. That is what they are. A screenshot in marketing is a factual assertion about current behaviour, and the only thing standing behind it is somebody remembering to retake it.