What is a soft 404, and why does it cost you rankings?

A soft 404 is a page that tells the human one thing and the machine another. The body says the page does not exist. The status line says 200 OK. Only one of those is load-bearing, and it is not the one your visitors read.

Crawlers do not read your copy to decide whether a request succeeded. They read the status code. A 200 means: this is a real page, index it, come back later to see if it changed.

How sites end up serving them

Rarely on purpose. The common routes are all framework behaviour:

• A catch-all route that matches every path and renders a not-found component without setting the status. The component is right; the response is not.
• A missing page redirected to the homepage, which returns 200 for a URL that was asking for something else entirely.
• A category or search page that renders successfully with zero results - technically a valid page, indistinguishable from an empty one.
• A static host configured to serve `index.html` for any unmatched path, which is how single-page-app hosting works by default.

Why the cost compounds

One soft 404 is harmless. The problem is that a catch-all producing them does not produce one - it produces an unbounded set. Every typo in an inbound link, every stale URL from a site redesign, every probe for `/wp-login.php` becomes a 200-OK page on your domain.

Google then does what it does with any 200: it crawls them, considers indexing them, and schedules them for recrawl. On a domain with a small crawl allowance, that budget is taken directly from the pages you wanted crawled. Sites with hundreds of URLs stuck in "Discovered - currently not indexed" very often have a catch-all quietly manufacturing competitors for the same budget.

There is a second cost. Every one of those URLs renders the same body - your not-found page, or your homepage. That is a large set of near-identical pages on one domain, which is exactly the pattern duplicate-content handling exists to suppress.

Finding them

Request something that cannot exist and read the status, not the page:

curl -s -o /dev/null -w '%{http_code}\n' https://example.com/asdfqwerty-not-a-page
curl -s -o /dev/null -w '%{http_code}\n' https://example.com/llms.txt

The first must return 404. The second returns 404 if you do not serve the file - which is the correct answer, and far better than 200 with your homepage in the body, because an auditing tool fetching that URL will parse the HTML it got back as though it were the file it asked for and report nonsense about its contents.

In Search Console, Pages → "Soft 404" lists what Google has already caught. Treat that as a floor, not a total: it only lists URLs Google happened to try.

Fixing it

The fix is always the same shape - make the status line agree with the body.

• In Next.js App Router, `notFound()` sets a 404 when the route is dynamic. If your dynamic segment enumerates its own valid values at build time, set `dynamicParams = false` so anything outside that set 404s automatically.
• For SPA hosting, configure the fallback to serve your 404 document with a 404 status rather than `index.html` with a 200.
• Do not redirect missing pages to the homepage. Serve a 404 that links to the sections that do exist.
• For genuinely removed content, 410 Gone is stronger than 404 - it tells Google the removal is deliberate and permanent.
• An empty results page is only a soft 404 if it is indexable. Add `noindex` to zero-result pages and let the 200 stand.

Our SEO checker requests a URL it knows cannot exist on your domain and compares the status line to what the body claims, so a catch-all returning 200 shows up as a finding rather than as an unexplained gap between the pages you published and the pages Google indexed.