What an llms.txt file should contain to be useful

llms.txt is a plain text file at the root of a site that tells language models which pages are worth reading and what each one covers. It is a routing hint, not a ranking signal: it will not make a model cite you, but it removes the guesswork about which page answers a question.

The most common mistake is a file that repeats the navigation — Home, About, Contact — and omits the pages that actually answer things.

What a useful file contains

• A one-line description of the site, phrased the way someone would ask about it.
• The pages that answer questions, each with a short "this page answers…" sentence.
• Stable, absolute URLs — not anchors, not redirects.
• Nothing that is not a page. No marketing paragraphs, no lorem.

# Aeolith

> Fifteen checks that decide whether an AI assistant can read, understand and quote a site,
> with the evidence behind each verdict.

## What the checks mean
- https://aeolith.com/standard: the full definition of all fifteen items and what counts as evidence
- https://aeolith.com/methodology: how a check is measured, and what the report records

## Fixing a failure
- https://aeolith.com/repair-guide: the fix for each failed check, per stack

## Optional
- https://aeolith.com/articles: longer explanations of the failures we see most often

Two things that matter more than the format:

1. Every listed URL must return 200. A dead link in llms.txt is the same as a dead link anywhere else — and our own check A4 (dead-link pollution) fails us for it.
2. The description must match the page. If the file says a page explains X and the page is about Y, the model learns to distrust the file.

Verify it in one command

curl -sS https://example.com/llms.txt | grep -o 'https\?://[^ )]*' | while read -r u; do
  printf "%s %s\n" "$(curl -s -o /dev/null -w '%{http_code}' "$u")" "$u"
done

Anything that is not 200 gets removed or fixed before you call the file done.

Where it fits in the fifteen checks

llms.txt is one of the fifteen items (check A5 alongside the sitemap and stability group). It is deliberately a small part of the score: a site with excellent raw HTML and valid structured data scores well without it. Treat it as a hint you add once the foundation is in place, not as the foundation.