CogniPrep builds practice for real hiring assessments, which means almost every
fact on the site belongs to a vendor: how many questions, how long, what counts
as a pass. A lot of those facts are not published anywhere. The vendor sells to
employers, not to candidates, and has no reason to put an item count on the
internet.
So the interesting engineering question is not "how do we find out". It is "what
do we do with a fact we cannot get", and the answer has to survive the end of
the project that discovered the gap.
The plan is disposable
Work here happens in rounds, and a round starts with a contract document:
scope, decisions, sources, what we refuse to build. When the round merges, the
document is deleted. The most recent one was 242 lines and lasted about a week.
Over the project's life, 32 markdown files have been deleted out of docs/.
Three remain. The surviving backlog explains how to find the dead ones in its
own header:
Open items carried over from finished rounds, checked on 19 Nov 2026. Their
round contracts were deleted once merged;git log --diff-filter=D -- docs/
finds them. Remove an item when it is done.
That is 131 lines, seven sections and 36 items for a project with 491 practice
tests and 1,155 public URLs. It is small because almost everything a plan
contains stops being true the moment the work ships. Scope becomes code.
Decisions become comments next to the thing they decided. Sources become
citations on the page that uses them.
Three kinds of thing cannot be pushed into the code, and those are the sections
that exist.
The register of things we do not know
The best of them is headed "Unconfirmed facts behind live games", and under the
heading is the entire rule:
No page states these as fact; replace them when a source appears.
Thirteen providers have an entry. Five were added last weekend. Abbreviated,
they read like this:
- Saville: the full tests' item counts beyond the 2017 product pages; whether Checking gives partial credit. - AMN: every time limit (AMN publishes none), skip and return rules, real part order. - Ramsay: option counts, whether candidates can go back, calculator rules. - CBP: current item counts, and the scale behind the pass mark of 70. None of that is work waiting to be done. It is work that cannot be done, by us,
until somebody publishes something. Which is exactly why it has to be written
down somewhere permanent: the alternative is that the knowledge lives in the
head of whoever researched that provider, and the next person to touch the page
fills the gap with a confident sentence.
The register has an output you can see
A list of unknowns is worth nothing if the pages pretend otherwise. So the rule
has a visible form: where a vendor publishes no figure, the field says so.
Counting across the page registry:
"Not published" 347 occurrences in 57 files "Reported" 123 "our estimate" 52 Three labels, three different epistemic states. "Not published" means the vendor
has never said. "Reported" means somebody other than the vendor says it, usually
a candidate forum or a rival prep site, and we are passing on the claim with its
owner attached. "Our estimate" means we picked a number to build with and we are
telling you it is ours.
The failure mode all three avoid is the one that makes prep sites untrustworthy:
a number with no owner. Rival pages in this market routinely state item counts
and time limits as bare fact for tests whose publishers have released neither.
The sharpest example we have on file is a battery section that its own agency
removed in February 2024, which three prep sites were still describing and one
was still selling drills for when we re-checked them this month.
And it is where a "no" goes
The last use of a permanent backlog is to stop a decision being re-litigated
every round. Last week a provider was researched in full, a dossier written, a
two-game plan drawn up, and then not built. The reason is one entry:
BKSB (
docs/providers/bksb.md): held on 20 Nov 2026 because it is a
placement test with no pass mark, not a test you pass to get the job. The
dossier's two-game plan is ready if that changes.
Without that line, the same provider comes back in three months as a bright
idea, and the work of deciding against it happens twice. With it, the research
is not wasted: 78 provider dossiers sit in the repository, 94,608 lines of them,
and the ones that did not become products are the cheapest part of the next
decision.
The honest caveat is that this kind of backlog rots differently from a task
list. Its items are keyed on somebody else's publication state, which changes
without telling you, so an item leaves the list when a person goes and looks
again, not when a person does work. Hence the check date in the header, and
hence the fact that it is a date rather than a promise.
See it
The register is internal. Its consequences are not.
- Open /games/saville/error-checking and read the facts panel. Against "Questions" it says "Not published for the full test", and against "Time limit" it gives the one figure the vendor does publish, for a different test, and says the full test's is not published. Those two fields are the Saville backlog entry, rendered.
- Open /games/amn/rekensommen, a Dutch test whose publisher releases no timings at all. "Time limit: Not published. No countdown is shown." The clock our own practice uses appears further down the page, where it is labelled as our estimate rather than as the test's.
- Count them yourself on any test page:
curl -s https://cogniprep.app/games/saville/error-checking \ | grep -o 'Not published' | wc -l You will get double what the page shows, because a Next.js page ships its
text twice, once as HTML and once in the payload the client hydrates from.
- Then try the same grep on any competitor's page for the same test and see what comes back.
If you publish facts about somebody else's product, the thing to steal here is
not the backlog file. It is the pairing: a permanent register of what you do not
know, and a rendered label for each kind of not-knowing, so that the register
cannot quietly stop being true on the pages.
