Skip to content
WebsiteValidator
All posts
SEO5 min read

Why do link checkers report broken links that work fine?

Because a large share of the web answers automated clients with 403 or 429 while serving humans normally. That is bot protection, not a broken link.

By · Arcetis

Because a large share of the web answers automated clients with 403 or 429 while serving humans perfectly. LinkedIn, Instagram, X and most news sites all sit behind bot protection that refuses anything without a browser fingerprint - and a naive checker reports every one of those as broken.

That is not a minor annoyance. A report full of false positives trains you to ignore it, which means the one real 404 in your navigation gets ignored too.

What should a checker distinguish?

  1. 1**Broken** - 404 or 410. The destination explicitly says the resource does not exist. This is the only category worth scoring.
  2. 2**Server error** - 5xx. The destination is failing right now, which may be transient, and is worth telling you about at lower weight.
  3. 3**Unverified** - 401, 403, 429, 405, LinkedIn's 999, or a timeout. Your checker was refused. This is information about the checker, not the link.

Conflating the third category with the first is what makes most link reports untrustworthy.

Why does the checker's own behaviour matter?

Because it can cause the failures it reports. Checking sixty links at high concurrency looks like a small scraping attack, and the most likely response is that the target starts returning 429 - at which point your report fills with false failures caused entirely by your own tool.

Low concurrency and a retry with backoff are not politeness features. They are what keeps the results honest.

What about links a checker genuinely cannot resolve?

It should say so and stop there. A checker that has requested 50 of a page's 188 links and reports 'no broken links' is making a claim it has not tested. Stating what was checked and what was skipped is the difference between a result and a guess.

How do I check my own links?

The broken link checker on this site requests every link on the page, sorts the results into the three categories above, and scores only genuine 404s - reporting the rest so you can judge them.

Frequently asked

Why does my link checker say a working link is broken?
Because the destination refused an automated request. Cloudflare, Akamai and most WAFs answer bots with 403 or 429 while serving browsers normally. LinkedIn returns a non-standard 999 for the same reason. None of those is a broken link.
What status codes actually mean a link is broken?
404 and 410. Both are the destination stating the resource does not exist. A 5xx means the destination is failing, which may be transient, and a 401/403/429 means your checker was refused.
Should I remove links that a checker cannot verify?
No. Open them in a browser first. Removing a working LinkedIn or Instagram link because a bot was blocked is a real cost paid for a false positive.

Check this on your own site.

Run the check

Audit your site in 40 seconds

986 deterministic rules across eleven modules. Every finding comes with evidence, a citation and a fix preview.

Run a free scan

No signup. No credit card. Nothing stored but the result.