Why do link checkers report broken links that work fine?
Because a large share of the web answers automated clients with 403 or 429 while serving humans normally. That is bot protection, not a broken link.
By Sapun Lamichhane · Arcetis
Because a large share of the web answers automated clients with 403 or 429 while serving humans perfectly. LinkedIn, Instagram, X and most news sites all sit behind bot protection that refuses anything without a browser fingerprint - and a naive checker reports every one of those as broken.
That is not a minor annoyance. A report full of false positives trains you to ignore it, which means the one real 404 in your navigation gets ignored too.
What should a checker distinguish?
- 1**Broken** - 404 or 410. The destination explicitly says the resource does not exist. This is the only category worth scoring.
- 2**Server error** - 5xx. The destination is failing right now, which may be transient, and is worth telling you about at lower weight.
- 3**Unverified** - 401, 403, 429, 405, LinkedIn's 999, or a timeout. Your checker was refused. This is information about the checker, not the link.
Conflating the third category with the first is what makes most link reports untrustworthy.
Why does the checker's own behaviour matter?
Because it can cause the failures it reports. Checking sixty links at high concurrency looks like a small scraping attack, and the most likely response is that the target starts returning 429 - at which point your report fills with false failures caused entirely by your own tool.
Low concurrency and a retry with backoff are not politeness features. They are what keeps the results honest.
What about links a checker genuinely cannot resolve?
It should say so and stop there. A checker that has requested 50 of a page's 188 links and reports 'no broken links' is making a claim it has not tested. Stating what was checked and what was skipped is the difference between a result and a guess.
How do I check my own links?
The broken link checker on this site requests every link on the page, sorts the results into the three categories above, and scores only genuine 404s - reporting the rest so you can judge them.