You click the link. It opens. The report still says broken. Nothing is wrong with your eyes and usually nothing is wrong with the link: the server answered you and refused the checker, and the two arrive looking almost identical.
The short version
A link checker is a program asking a web server a question. A browser is a person asking the same server the same question. Increasingly, servers give those two askers different answers.
When a checker gets refused, what comes back is a status code that is not 200 OK. So is what comes back from a page that has genuinely been deleted. On the face of it they look the same: both are a failure to fetch the page. The reasons behind them are nothing alike, and the number alone does not sort them out.
Why servers refuse the checker but serve you
None of this is malice. Each of these is a real defence doing its job.
- Bot filtering. Large platforms block automated traffic on purpose. Your request carries a browser, a history, cookies and a mouse. The checker's request carries none of that, and gets shown the door.
- A firewall in front of the site. A content delivery network or a security service may issue a challenge that a browser passes silently and a checker cannot pass at all. The checker sees a page saying enable JavaScript and cookies to continue, which is not the page it asked for.
- Rate limiting. Check forty links on the same domain in a minute and the fortieth gets refused, even though the first one was fine. The link is not broken. You are.
- The server dislikes the method. A checker usually asks
HEADfirst, because it only wants the status and not the whole page. Some servers do not answerHEADat all, and reply with an error to a question they would have answered had it been phrased asGET. - Your own server is the one being blocked. The check leaves from wherever the tool runs. If that address has a poor reputation, or sits in a range someone has blocked, the refusal has nothing to do with the destination at all.
How to tell a refusal from a dead page, in ten seconds
Take the status code the report gives you. If it does not give you one, that is worth knowing on its own, because without it there is nothing to reason about.
| What you see | What it usually means | Is the link broken? |
|---|---|---|
404, 410 | The page is genuinely not there. 410 means the server is certain it is gone for good. | Yes. Fix it. |
403 | Forbidden. The server understood and decided not to answer. Almost always bot filtering. | Usually no. |
999 | Not a real HTTP code. It is a private "no" invented by a large network to turn away automation. | No. |
429 | Too many requests. You asked too fast. | No. Slow down and ask again. |
503, 502, 504 | The far side is unwell or behind a challenge, right now. | Not yet. Ask again later. |
405 | Method not allowed. The server will not answer HEAD. | No. A GET would have worked. |
| A timeout | Nobody answered in time. Slow server, or a check that gave up early. | Unknown. That is the honest answer. |
200 on a page that says "not found" | A soft 404. The page is missing but the server says it is fine. | Yes, and no status code will tell you. |
The second-to-last row is the one worth sitting with. "I could not check this" and "this is broken" are different findings, and collapsing them into one word leaves the reader guessing which of the two they are looking at. The last row is the opposite trap: a status code alone cannot catch every dead page.
What to do about each kind
If it is genuinely dead
Find the new address if there is one, and replace it everywhere it appears rather than in the one post you happened to be looking at. If the page is gone for good, decide between removing the link and pointing it at an archive copy.
If the check was refused
Do nothing to the link. There is nothing wrong with it. What you want is for the tool to stop telling you about it: a good report separates these out by itself, and a list of domains that never answer robots should be something you can set once and forget.
Checking that link by hand every month is not a solution. Neither is deleting a working link to make the warning go away. That costs you a link you meant to keep, and it is the quiet way a list of false alarms ends up shaping your site.
If you are not sure
Open it. You are the control group. If it opens for you and the checker says 403, you have your answer and the tool is describing its own experience, not the page.
Check one link, right now
If you have a link in front of you that you are arguing with, you can settle it here without installing anything. It shows you the status code, the redirects it followed and what the server actually said.
Check a single link → Free, no account.
Related
- Does a 403 mean the link is broken?
- Sites whose links always look broken
- Will a link checker change my site?
Doing this across a whole site
Onsite Link Verdict is our WordPress plugin for this. It checks every link on your site from your own server, and it will not call a link broken without showing you the evidence: the status, the redirects and the reason, in plain language. Refused checks live in their own list, described as what they are, because that is the distinction the whole job turns on.
It is free on the WordPress.org directory and free for good, not a trial.
Originally published on onsitetools.dev.
