My link checker said six of our citations were dead. All six were my HTTP client.

Our content pages cite their sources. A page that tells a candidate how long a test is has a link to the vendor page that says so, and when the vendor deletes that page we are left asserting a number with nothing behind it. So link rot is not a tidiness problem here, it is an accuracy problem.

I pulled every external <a href> out of all 530 live pages: 324 distinct URLs across 140 hosts, carried by 121 of the 530 pages. 86 of those hosts appear exactly once. 22 links point at web.archive.org, 37 at a government domain. The most-cited single page carries 12 outbound citations.

Then I checked them, three times, and the only thing the first two passes measured was my own tooling.

Pass one: Python, HEAD first, ten at a time

for method in ("HEAD", "GET"): try: r = urllib.request.urlopen(Request(u, headers={"User-Agent": UA}, method=method), timeout=25) return {"status": r.status} except urllib.error.HTTPError as e: if method == "HEAD" and e.code in (403, 405, 400, 501): continue # retry as GET  return {"status": e.code} 

Six dead links. That is a tidy finding to take to the content owner, and it is wrong.

Pass two: curl, GET only, one request per second

curl -s -o /dev/null -L --max-time 30 -w "%{http_code}" -A "$UA" "$url" 

All six 404s came back 200.

Five of them are Test Partnership's own public example tests, which we cite on five per-test pages:

https://www.tptests.com/candidate/practice?accesskey=InductiveExample -> 404 to HEAD, 200 to GET 

The sixth is a hiring-process page on jobs.exeloncorp.com, same behaviour. Those hosts answer a HEAD request with a 404 and the identical URL with a 200 to GET. Look at my retry ladder again: I retried as GET for 403, 405, 400 and 501, and treated 404 as authoritative. 404 is the one code you would never think to retry, and it is the one that needs it, because an app framework that never implemented HEAD will often answer it from a catch-all.

Four of the six 429s also became 200. They were web.archive.org, and ten concurrent workers is enough to rate limit yourself against it. Two archive-hosted images stayed 429 at one request per second as well, which is the Archive's own throttle rather than a missing file.

Rule one: use GET. Rule two: your concurrency is an input to the result.

Pass three: a real browser, for the ones a client cannot settle

Twelve URLs still answered 403 after the curl pass, and three answered 502. I opened each in a real Chromium session, which is the only client whose verdict matches what a reader experiences.

A 403 from a bot wall and a 403 from a deleted page look the same in a spreadsheet and are opposite findings. The ICO's is the clearest case: the page is fine, and the "security verification" page in front of it answers 403 to anything without a browser fingerprint.

The bottom three rows are the honest residue. post.ca.gov closes the connection before TLS completes, to curl, to Python and to Chromium alike, from this network. I cannot say whether those four citations are good; I can only say I cannot check them from here. That is a different sentence from "the link is dead", and writing the first one when you mean the second is how a correct citation gets swapped out for a worse one.

What the audit actually found

Of 324 external citations: zero confirmed 404 or 410. Three are PDFs on a host that is currently in scheduled maintenance, which is a wait, not a fix. Six are unverifiable from this machine. The remaining 315 resolve.

The one real content defect it surfaced is not a status code at all. One of the Test Partnership demo links reads:

?accesskey=critcal-verbal-28836 

It loads, because the access key is whatever their system says it is, but "critcal" is sitting in a published URL on our site and nobody can tell from the outside whether it is their typo or ours.

If you build one of these

• GET, not HEAD, and retry a 404 at least once with GET before believing it.
• One request per host at a time. Your 429s are usually self-inflicted.
• Follow redirects, and record the final URL. A 200 after three hops to a homepage is rot wearing a 200.
• Classify, do not just count: bot wall, throttle, maintenance, network refusal, real 404. Only the last one is a content task.
• A real browser is the tiebreak, and it is worth the manual minute on the handful of URLs a client cannot settle.

See it

The two heaviest citation pages are good places to see what 12 outbound sources on one page looks like, and both are public:

• PELLETB practice, which carries the four post.ca.gov links above
• the Revelian cognitive ability test page, which carries the archive.org ones

Open either, view source, and grep for href="http. Then run the two-line version of my pass one against that page's links and watch it tell you something is broken.