aeolith Check a site
Article

Server-side rendering or prerendering: which one an AI crawler needs

The answer first: the text has to be in the HTML the server returns. Whether that HTML comes from server-side rendering, from a prerender step, or from a static build does not matter to a crawler. What matters is that curl sees the content — no browser, no scripts, no waiting.

A client-rendered single-page app returns a shell: a <div id="app"></div>, a bundle, and no words. Assistants that do not execute JavaScript read the shell and move on. This is check A2 (content in raw HTML).

Verify in one command

curl -sS https://example.com/ | python3 -c "
import sys, re
html = sys.stdin.read()
text = re.sub(r'<[^>]+>', ' ', re.sub(r'<(script|style)[^>]*>.*?</\1>', ' ', html, flags=re.S))
words = len(text.split())
print(f'{words} words without running JavaScript')
print('empty shell' if words < 50 else 'content is in the HTML')
"

Run it on the pages that matter — product pages, pricing, documentation — not just the homepage. A homepage is often statically generated while the pages that answer questions are not.

The three fixes, in order of cost

  1. Server-side render the pages that answer questions. Most frameworks support it; the work is real but it is once. This is the most robust option because there is no second build step to forget.
  2. Prerender at build or request time. Good when the app is not yours to change: render each URL to static HTML, serve that to every user agent. Keep the prerender in CI — a prerender that only runs locally is a prerender that will be stale in production.
  3. Dynamic rendering (serve different HTML by user agent). It works, but keep it boring: serve the same content, only a different rendering, and never show crawlers a page you would not show a person.

The traps that survive the fix

  • Content behind interaction. A tab, an accordion or a "load more" button hides the text from every crawler that does not click. If it matters, it belongs in the initial HTML.
  • Metadata injected by JavaScript. Title, description and JSON-LD count as content. They must be in the source too.
  • A framework upgrade that silently drops SSR. This is why A2 is measured on every run rather than assumed: the fix can regress without anyone touching the template that broke.

What the report shows

Check A2 names the URLs where the raw response was an empty shell, and records the text-node count from the raw response next to the count from the rendered page — so "renders empty" is a measurement rather than a guess. Check A5 covers the plumbing that makes rendering possible at all: response time, HTTPS, and stability under repeated requests.

Check my domain free See all 15 checks
Check a site