What the file is for
The llms.txt proposal describes a Markdown overview with links that help agents navigate useful site content. It is especially relevant to documentation where concise references can save a reader from sorting through navigation and scripts. This is a proposed convention, not a universal search indexing contract.
What Google actually requires
Google’s official guidance for AI features says the established Search technical requirements and SEO practices apply. A page needs to be indexed and eligible for a snippet to appear as a supporting link. It does not require a new AI text file or special schema. Meeting requirements still does not guarantee crawling, indexing or appearance.
That guidance supports treating llms.txt as optional. It does not establish that every other provider ignores it, and it does not prove a measured uplift from publishing one.
A useful experiment, with a boundary
If you maintain documentation, curate a short file linking to authoritative pages, keep it current and test it with a tool that explicitly reads it. Check whether that tool reaches the correct reference more easily. Record the tool, task and date. That is a navigation experiment; it is not an AI search ranking experiment.
- Link to accessible, accurate pages and concise references. Do not publish private information in a public file.
- Keep the same facts in the ordinary pages. An optional overview should not be the only place a key answer exists.
- Leave access policy in robots.txt and the appropriate page controls. llms.txt does not replace them.
- Do not populate the file with fabricated awards, endorsements or instructions to manipulate a recommendation.
What to repair first
Look for blocked search crawlers, accidental noindex directives, incorrect canonical URLs and essential text hidden behind client-side rendering. Then make the page’s purpose and answers clear. Structured data should describe the visible facts rather than invent signals. These changes are inspectable even when an answer engine’s behavior remains uncertain.
How this audit handles it
We report whether llms.txt exists as an optional observation. It carries zero score weight. A site without it loses no points; a site with it gets no ranking promise. Our HTML heuristics assess a technical checklist, while answer samples remain a separate source of evidence.
In MOCK mode both the website and answer results are synthetic fixtures. They validate the local workflow, not the effect of llms.txt on any engine. Repeated real observations would need authorized official API use and a controlled prompt set.
Sources reviewed on 2026-10-02: the llms.txt proposal and Google Search Central’s AI features documentation. We have not run a causal ranking experiment. Adoption by a documentation tool and appearance in an AI answer are separate outcomes.