What is llms.txt? Format, example and whether it works
llms.txt is a proposed Markdown file that gives AI tools a short, curated map of a site. It can help a tool that chooses to read it. It is not a ranking signal, an access control or a way into AI search answers, and none of the major AI search operators documents using it to choose sources.
The short answer
- What it is: a Markdown file at
/llms.txtthat lists a site's most useful pages with short notes. Jeremy Howard proposed it on 2024-09-03; the current version of the proposal was updated on 2026-08-10. - Does it help SEO? Google's guidance for AI Overviews and AI Mode says you don't need new machine-readable files, AI text files or markup to appear in them.
- Does it help AI search? We found no documentation from Google, OpenAI, Anthropic or Perplexity saying their search systems use llms.txt to choose, rank or cite sources. Their crawler documentation talks about robots.txt instead.
- Where it can help: documentation, API references and help centres that coding assistants and agents read. Generated automatically, it costs almost nothing to keep.
The format
The proposal at llmstxt.org defines a small, ordered structure:
- An H1 with the name of the site or project. This is the only required part.
- A blockquote with a short summary containing the key information needed to understand the rest of the file.
- Optional detail as paragraphs or lists, with no headings.
- H2 sections of links, each entry written as
- [name](url): notes, where the notes are optional. - A section titled Optional for secondary links that an agent can skip when it needs a shorter context.
The file can sit at the root, /llms.txt, or under a path such as /docs/llms.txt, where it covers the URLs beneath that path. When more than one file applies, agents should use the most specific one.
The proposal also suggests offering a Markdown version of useful pages at the same URL with .md added, such as page.html.md, or with the extension replaced, such as page.md. A page can point to its Markdown version with rel="alternate" type="text/markdown" and to the llms.txt file that covers it with rel="describedby", either as HTML link elements or as HTTP Link headers. llms-full.txt is not part of the proposal; it is a convention some documentation platforms use for a single file with the full text.
An example
A small business site could publish something like this:
# Example Dental Studio
> Family dental practice in Leeds offering check-ups, hygiene, implants and emergency appointments, open Monday to Saturday.
New patients can book online. Prices on the fees page include VAT.
## Services
- [Check-ups and hygiene](https://example.com/services/check-ups): what an exam includes and how long it takes
- [Dental implants](https://example.com/services/implants): process, timeline and fees
## Practical
- [Fees](https://example.com/fees): current price list
- [Book an appointment](https://example.com/book): online booking and phone number
## Optional
- [About the team](https://example.com/about)
- [Patient reviews](https://example.com/reviews)CiteVisor publishes its own llms.txt, generated at build time from the guides, tools and industry playbooks, so it never drifts from the pages it describes.
llms.txt, robots.txt and sitemap.xml compared
| File | What it does | Who reads it | Controls access |
|---|---|---|---|
robots.txt | Grants or refuses crawling, per user agent | Search engines and AI crawlers that follow the standard | Yes, for crawlers that obey it |
sitemap.xml | Lists the URLs you want discovered, with dates | Search engines | No |
llms.txt | A curated Markdown map with short notes | Tools that choose to read it | No |
Access decisions belong in robots.txt and your firewall. Our AI crawler list and robots.txt generator cover the crawler side.
Does llms.txt work? What the evidence shows
Google. Its documentation for AI features says you don't need to create new machine-readable files, AI text files or markup to appear in AI Overviews or AI Mode, and that no special schema.org markup is required. A page needs to be indexed and eligible to show a snippet. Google's guide to optimizing for generative AI features, last updated 2026-07-10, is blunter: Google Search itself doesn't use these files, and maintaining an llms.txt file will neither harm nor help a site's visibility or rankings in Google Search. That settles the Google question: llms.txt is optional there.
AI search operators. OpenAI, Anthropic, Perplexity and Google document how their crawlers handle robots.txt. We found none that documents reading other sites' llms.txt files to choose, rank or cite sources. That is not proof that no system reads them; it means there is no documented benefit to count on.
Who publishes one. We requested /llms.txt from several sites on 2026-10-21:
| Site | Result |
|---|---|
| developers.openai.com (OpenAI developer docs) | Published |
| platform.claude.com (Anthropic developer docs) | Published |
| docs.perplexity.ai (Perplexity API docs) | Published |
| developers.cloudflare.com, docs.stripe.com and vercel.com | Published |
| developers.google.com | Not found (404) |
| www.anthropic.com | Not found (404) |
In this sample, the files sit on developer documentation, which coding assistants and agents read on purpose; Anthropic's main site and Google's developer site had none. Publishing a file for your own docs is a different thing from your search crawler using everyone else's.
Your own logs. The most direct evidence is whether anything requests your file. In a combined access log, the user agent is the sixth field when you split on double quotes:
grep "GET /llms.txt" access.log | awk -F'"' '{print $6}' | sort | uniq -c | sort -rnIf no AI crawler ever asks for the file, it is not doing anything for AI search on your site.
When it is worth publishing
- Documentation and developer products, where people point coding assistants and agents at your docs. A good map saves them from wading through navigation, scripts and outdated pages.
- When it is generated automatically from the same source as your pages, so it cannot go stale.
- As a navigation experiment: curate a short file, test it with a tool that explicitly reads llms.txt, and record the tool, the task and the date. That tests navigation, not AI search ranking.
It is not worth prioritising over the work that does decide visibility: allowing the right search crawlers, removing accidental noindex directives, fixing canonical URLs and putting the main text in the initial HTML. The GEO audit checklist covers those.
Practical rules
- Link only to public, canonical pages, and never to private or internal URLs.
- Give each link a one-line note that says what the page answers.
- Put secondary pages under
## Optional. - Keep every fact on the real pages too; llms.txt should never be the only place an answer exists.
- Regenerate the file whenever the site deploys.
- Serve it as UTF-8 plain text or Markdown.
- Leave out instructions aimed at models and claims you could not defend on the page itself, such as awards or endorsements.
How CiteVisor treats llms.txt
Our audit reports whether llms.txt exists as an optional observation with zero score weight. A site without one loses no points, and a site with one gets no ranking promise. The checks that carry weight are the ones with documented effects: crawler access, indexing signals, canonical URLs, structured data that matches the page, and content present without JavaScript. Run the free AI visibility checker to see them for your own page.
Sources
- llms.txt proposal
- Google Search Central: AI features and your website
- Google Search Central: Guide to optimizing for generative AI features
- OpenAI developer documentation llms.txt
- Anthropic developer documentation llms.txt
- Perplexity documentation llms.txt
Reviewed on 2026-10-21. We have not run a causal ranking experiment; publishing a file and appearing in an AI answer are separate outcomes.
Frequently asked questions
What is llms.txt used for?
It gives AI tools and agents a short Markdown overview of a site, with links to the pages that matter most, so a tool that reads it can find the right page without working through navigation and scripts. It is most useful for documentation.
Does Google use llms.txt?
No. Google's guide to optimizing for generative AI features says Google Search itself doesn't use these files, and that maintaining one will neither harm nor help your visibility or rankings in Google Search. To appear in AI Overviews or AI Mode, a page must be indexed and eligible to show a snippet.
Do ChatGPT, Claude or Perplexity read llms.txt?
OpenAI, Anthropic and Perplexity publish llms.txt files for their own developer documentation, but we found no documentation saying their search crawlers use other sites' llms.txt files to choose or rank sources. Check your own logs for requests to /llms.txt.
What is llms-full.txt?
A convention some documentation platforms use for one file containing the full text of the docs. It is not part of the llms.txt proposal, which links to separate Markdown pages instead.
Can llms.txt block AI training or AI crawlers?
No. llms.txt contains no access rules. Use robots.txt groups for each crawler, such as GPTBot or Google-Extended, and firewall rules where you need enforcement.
Will a missing llms.txt lower my CiteVisor score?
No. Our audit reports llms.txt as an optional observation with zero weight. A site without one loses no points, and a site with one gets no ranking promise.