Does llms.txt matter? What the data says in 2026
By Alex Bouchard · 6 September 2026
llms.txt is a proposed convention: a Markdown file at your site root that summarises the site for language models. It is easy to add, which is why a lot of people have. Whether any AI system reads it is a separate question, and it has an answer now.
Adoption is up. Requests are not.
Originality.ai's tracking across more than three million sites counted 36,120 llms.txt instances by May 2026, an 8.8x increase over twelve months. Ahrefs then looked at what actually fetched those files, using server logs from 137,000 domains: 97% of llms.txt files received zero requests in May 2026. Of the requests that did arrive, SEO audit tools were the largest source at 21.7%; GPTBot accounted for 4.51%, ClaudeBot for 0.80%, and AI retrieval bots as a group for about 1.1%. That is the picture of a file being written far more often than it is read.
The vendors' own positions match the logs. Google has said llms.txt will not positively or negatively affect visibility or rankings and is not required for Search. OpenAI's crawler documentation does not mention it and points site owners to robots.txt. Anthropic takes the same line.
Why it cannot fix the problems that matter
Before an AI system can cite a page, four things have to be true, in order: the crawler gets a 200 from your edge rather than a challenge; the cold response comes back before the crawler gives up; the words are in the raw HTML rather than rendered by JavaScript; and robots.txt allows the search crawler in question. A Markdown summary at the root does none of those. If the homepage is blank without JavaScript, the model that reads your llms.txt — if one ever does — still cannot read the pages it points to.
Our scan reports whether an llms.txt exists as an informational line, not a score, for exactly this reason. It is the last item on the list, and it is optional.
If you add one anyway
Keep it short and true: what the site is, the handful of pages that matter, and a line about who to contact. Serve it at /llms.txt as plain text and let it be. Then spend the saved afternoon on the four gates above, because those are what the crawlers in your logs are actually testing.
Sixty seconds, no signup: fetch your homepage the way GPTBot, ClaudeBot and PerplexityBot fetch it, count the words that survive, and read what your robots.txt actually tells each AI bot.
Scan free →Sources
- PPC Land — llms.txt adoption rises 8.8x but 97% of files get zero AI requests (Originality.ai + Ahrefs, July 2026)
- OpenAI — Overview of OpenAI crawlers (no llms.txt guidance)
- Anthropic — Does Anthropic crawl data from the web? (robots.txt guidance)
- ReadableByAI — what the scan checks, and in what order
Questions people ask
Do ChatGPT, Claude or Perplexity read llms.txt?
None of them document it. OpenAI's and Anthropic's crawler documentation direct site owners to robots.txt and do not mention llms.txt. In Ahrefs' May 2026 server-log analysis across 137,000 domains, 97% of llms.txt files received zero requests, and AI retrieval bots accounted for about 1.1% of the requests to the files that were fetched.
Does llms.txt help Google rankings?
Google has said llms.txt will not negatively or positively impact visibility or rankings and is not required for Google Search.
Is there any downside to adding llms.txt?
Only opportunity cost. It is a plain text file. The risk is treating it as the fix when the site is blank without JavaScript or blocks the search crawlers at the edge — those are what AI systems actually hit.
What should I do before llms.txt?
Confirm the crawlers get a 200 from your edge, that your cold response is fast, that your words exist in the raw HTML, and that robots.txt allows the search crawlers you want. Our free scan checks all four.
Sixty seconds, no signup: fetch your homepage the way GPTBot, ClaudeBot and PerplexityBot fetch it, count the words that survive, and read what your robots.txt actually tells each AI bot.
Scan free →Corrections: every claim on this page is attributable to vendor documentation or to our own published measurement. If one is wrong, tell us and it gets fixed with a dated note.