AI crawlers cannot read my Next.js site — what to change

On Next.js, a page can be readable in a browser and invisible to an AI crawler. Here is what usually causes it, what to change, and what you cannot change from your side.

What usually causes it

Routes that quietly became client-rendered (heavy use client components, data fetched after hydration) while the marketing pages stayed static. Metadata injected at runtime instead of exported.

How to confirm it on Next.js

curl -sS -A "GPTBot/1.0 (+https://openai.com/gptbot)" -o /tmp/p.html -w "%{http_code} %{size_download}\n" https://example.com/
python3 -c "import re;h=open('/tmp/p.html').read();print(len(re.sub(r'<[^>]+>',' ',h).split()),'words in raw HTML')"

A 403 is a block; a small word count is a rendering failure. They are fixed in different places, so confirm which one you have before changing anything.

What you cannot fix on Next.js

Verify each route with curl rather than the browser, force static or server rendering for the routes that answer questions, and export metadata through the framework's own API so it lands in the HTML.

Where this fits

Crawler access is A1 and content in the raw HTML is A2 — both in group A, which runs first because nothing downstream matters if the request is refused. The full definition is in the standard, and the order that avoids redoing work is in the repair guide.