My catalog has 4,323 AI tool listings. On 4,176 of them (96.6%) the "About" block ends with a sentence I never wrote:

Voicepanel is a paid AI tool in the Business category. It currently ranks #3040 in the Business category.

The number is copied by a crawler from the directory I pull records from. It is the only "objective" ranking signal sitting on my pages, and readers see it 4,176 times a day's worth of page loads. Last week I asked a simple question: is that sentence still true today? I re-fetched the 12 source pages behind 12 of my listings and read the badge again.

All twelve numbers had moved

Every one. Not one matched what I publish.

listing number I publish number at the source now badge's own list, then vs now
QuickMagic #1031 #1039 Video Generators → Latest AI
UltraPixel AI #1321 #1331 Github Projects → Latest AI
PostWizard AI #1786 #1806 Marketing → Latest AI
Chatter by Hume AI #1882 #1906 Text To Speech → Latest AI
Cuebric #2226 #2246 Image Generators → Latest AI
Code Llama #2368 #2392 Assistant Code → Latest AI
GaussianObject #2561 #2590 3D model → Latest AI
Voicepanel #3040 #3070 Business → Latest AI
Illustroke AI #294 #298 Image Generators → Image Generators
Maze AI #295 #299 Image Generators → Image Generators
Iconik AI #301 #305 Image Generators → Image Generators
PatternAI #302 #306 Image Generators → Image Generators

The deltas are all positive: +4, +4, +4, +4, +8, +10, +20, +20, +24, +24, +29, +30. Nothing moved up the list. That is what an append-only ranking does to everyone below the new entries. A stored rank is not a position, it is a timestamp — and I record no timestamp next to it.

The number and the category were never a pair

That is the part that surprised me. On 8 of the 12 source pages, the badge is no longer attached to a category at all; it reads "#3070 in Latest AI". "Latest AI" is a chronological feed of every entry the source has ever listed — a list of three thousand-odd items, not a subject.

My crawler already knows that feed is not a category. It has an explicit exclusion list:

const CURATION_CATEGORIES = new Set(["featured-en", "last-ai-en"]); const primaryIsCuration = primarySlug && CURATION_CATEGORIES.has(primarySlug); const categories = []; if (primarySlug && !primaryIsCuration) { categories.push({ name: primaryName || primarySlug, slug: primarySlug }); } 

So the category is dropped — but the rank is read from a different selector on the same badge span (rank: text(".aix_rank-num")) and is kept. The template then marries it to whatever survived as categories[0]:

const rankNote = record.rank ? ` It currently ranks #${record.rank} in the ${category} category.` : ""; 

The result is a sentence that asserts a position in "the Business category" for a number that was measured in somebody's "just added this week" river. The two halves came off the same DOM element and my code separated them, then re-joined them as if they still belonged together.

476 pages print a rank that cannot exist

If the sentence were a rank inside my own category page, it could never exceed the number of tools on that page. It does, 476 times.

  • The Business category on my site contains 233 tools. I publish "#3040" there.
  • Image Generators contains 269. I publish "#2226".
  • Video Generators: 294 tools, highest claim #1031.
  • Marketing: 164 tools, highest claim #1786.

And the reverse check is just as bad. My Video Generators listing holds 205 tools that name that category in their sentence; their claimed ranks fill only 133 of the 205 positions. Seventy-two slots are simply missing — because the ranks were sampled out of a much longer foreign list and I only ever scraped part of it. A real in-category ranking is dense: 1, 2, 3, no holes. Mine is sparse, which is the tell that it was never a ranking of my set.

So the "#1" claims are also not what they look like. 37 listings claim rank #1, in 37 different named categories. Each one is plausible as "first in some list at the source, on the day I scraped it"; none of them is first in my category, and my category pages don't even order by that field — they order by a mirrored read counter I also did not measure.

The same fetch found a pricing bug

I checked the pricing badge while I was there, since it is the other column readers filter on. Eleven of twelve matched. The twelfth: Cuebric's source badge says Free-Trial, and my row says free. The reason is a precedence chain, not a data problem:

if (pricingText.includes("freemium")) pricingType = "freemium"; else if (pricingText.includes("free")) pricingType = "free"; else if (/paid|payant|premium|subscription/.test(pricingText)) pricingType = "paid"; 

"free-trial" contains "free". A trial is a paid product with a demo, and it lands in the bucket a reader uses to avoid paying. That bucket is my biggest one (2,198 of 4,323 listings), so a substring rule with the wrong order is a trust event waiting to happen. One mismatch in twelve is a 8% sample rate, not a measurement of the whole library — the honest statement is "the classifier has a known hole and I have not sized it".

What would make the sentence true

Not "crawl more often". Recrawling just gives me fresher stale numbers. The sentence needs the triple it was cut from:

  1. Store the list with the number. rank: {value, listSlug, listSize, capturedAt}. If listSlug is a feed I don't publish, don't emit the sentence — that alone would remove the eight of twelve cases above.
  2. Or compute rank locally, over my own category pages, so "#4 in Video Generators" means position four in a list a reader can scroll, and it is dense by construction.
  3. Or delete it. I can also stop claiming a category the badge never named, and fix the pricing precedence with a word-boundary match on the exact source labels.

Until one of those lands, the cheapest honest version of the page is: the number goes away, and the read counter stays, clearly labelled as mirrored from the source. If you want to see what my "best" ordering actually is when nothing is pretending to be a rank, it is here: AIVerse — most-read AI tools. It ranks by a copy of somebody else's view count, which is a weaker claim than "#3040 in Business" and therefore a safer one to print.

What I'd take from this

  • A rank is only meaningful with the list it came from. If you can't print the list, you can't print the rank.
  • Compare the max rank in a category to the count of items in that category. It costs one query, and mine said 476 impossible claims.
  • Check whether your claimed ranks are dense. Holes mean you're quoting a sample as if it were a census.
  • Put a captured-at date on any field that can only move. In my sample all twelve deltas pointed the same direction, so a number without a date is a slowly-growing lie.

The numbers in this post come from a read-only pass over my local SQLite database (4,323 rows) and twelve source pages fetched today. Nothing here is fixed yet; this is the measurement, not the patch.