AI crawlers cannot read my WordPress site — what to change

On WordPress, a page can be readable in a browser and invisible to an AI crawler. Here is what usually causes it, what to change, and what you cannot change from your side.

What usually causes it

Three usual causes: an SEO plugin's "block AI" toggle writing a disallow rule, a security plugin's bot list, and a caching plugin serving a JavaScript shell or a challenge page to crawlers.

How to confirm it on WordPress

curl -sS -A "GPTBot/1.0 (+https://openai.com/gptbot)" -o /tmp/p.html -w "%{http_code} %{size_download}\n" https://example.com/
python3 -c "import re;h=open('/tmp/p.html').read();print(len(re.sub(r'<[^>]+>',' ',h).split()),'words in raw HTML')"

A 403 is a block; a small word count is a rendering failure. They are fixed in different places, so confirm which one you have before changing anything.

What you cannot fix on WordPress

Check the plugin toggle first (it is the most common and the least remembered), then test with curl instead of a browser; a cached challenge page looks like a healthy 200 until you read the body.

Where this fits

Crawler access is A1 and content in the raw HTML is A2 — both in group A, which runs first because nothing downstream matters if the request is refused. The full definition is in the standard, and the order that avoids redoing work is in the repair guide.