Category

Troubleshooting

Most sites fail the same handful of ways. Each page below starts from the symptom you can actually see, then gives the check to run and the fix for that stack.

9 pages in this category

Subcategories

• Crawler access

  The crawler never reaches your content: WAF rules, robots.txt or a CDN challenge.

• Markup and data

  The page is readable but says nothing a machine can use: missing or broken markup, titles, sitemaps.

• Rendering

  The crawler arrives but reads nothing: client-side rendering and JavaScript-only content.

Pages

• Sitemap submitted but pages are not indexed

  2026-09-28

• Google shows the wrong title or description

  2026-09-28

• Structured data is present but not parsed

  2026-09-28

• llms.txt is not working

  2026-09-28

• Next.js or a SPA: making it readable without JavaScript

  2026-09-28

• JavaScript content is not being read

  2026-09-28

• My site is invisible to AI assistants

  2026-09-28

• Cloudflare or a CDN is blocking AI crawlers

  2026-09-28

• GPTBot is blocked (and how to unblock it)

  2026-09-28

All articles