For two years, llms.txt has been the GEO industry's favourite recommendation: a small file, a quick win, an easy line on an invoice. Many audit tools still call a missing llms.txt "the #1 missing AI visibility asset."
We just down-weighted it in our own audit — and published the change in our methodology changelog. Here's the evidence that forced the call.
What the file was supposed to do
Proposed by Jeremy Howard in September 2024, llms.txt is a markdown file at your site root that summarises what your site is and links its most important pages — a curated map for AI systems, analogous to robots.txt but for comprehension rather than exclusion.
A reasonable idea. The question was always whether the engines would actually read it.
What happened instead
- Adoption stalled. Roughly a tenth of websites have one, and the curve is flat — nothing like the adoption trajectory of a standard that's working.
- Google has said publicly that it does not use llms.txt. No major consumer AI engine — not OpenAI, not Perplexity, not Anthropic — documents consulting the file for search or citation.
- Analyses that try to predict AI citations from site characteristics find llms.txt presence adds no predictive signal.
Meanwhile the one place llms.txt demonstrably IS consumed: AI developer tools. IDE agents like Cursor and Claude Code fetch it on documentation sites, where a curated map genuinely helps.
What we changed
In audit methodology v2026.08.2, llms.txt moved from a headline check to an informational one. Absence is no longer a meaningful penalty. Presence earns a small credit — the file costs ~30 minutes, is harmless, and helps AI dev tools. For documentation-heavy and developer-product sites, we still actively recommend it, because there the consumers are real.
The weight it gave up went where the evidence points: crawler access — including our new check for CDN-level AI blocking, which silently removes sites from AI answers far more often than a missing llms.txt ever did.
The bigger point
An audit that can't demote its own recommendations when the evidence turns is a checklist, not an instrument. We keep our checks tied to what engines demonstrably do — and when that changes, the changelog says so, dated and versioned. If your current GEO audit still leads with llms.txt, ask what else in it is two years old.