llms.txt is an emerging proposal — similar in spirit to robots.txt — that gives AI systems a structured guide to your most important content. It's not an official standard, no major LLM enforces it yet, but it's cheap to add and it's a useful authoring exercise even if no one reads it. Here's the honest take.
What llms.txt actually is
A markdown file at the root of your domain (e.g. /llms.txt) listing your most important pages, a one-line description of each, and grouped by topic. Optionally a longer /llms-full.txt with the actual content concatenated for direct consumption.
What it is not
It's not a blocking mechanism (use robots.txt and AI-specific user agents for that). It's not an official W3C or IETF standard. And it's not currently honoured by ChatGPT, Perplexity, Claude, or Gemini in any documented way.
Why we still recommend adding one
Three reasons. First, it's a five-minute job once you have the content map. Second, it forces you to articulate your site's information architecture from an LLM's perspective — which usually surfaces gaps you should fix anyway. Third, adoption is moving and being early costs nothing.
- Put it at /llms.txt at the domain root.
- Group links by topic with one-line descriptions.
- Keep it under 100 entries; include only your highest-value pages.
- Maintain it like a sitemap — update when the architecture changes.
Frequently asked questions
Where does llms.txt go?
At the root of your domain: https://yoursite.com/llms.txt — same convention as robots.txt.
Does ChatGPT read llms.txt?
Not in any documented way today. The proposal exists but enforcement is up to each LLM operator.
Does it replace robots.txt?
No. robots.txt controls what crawlers can access. llms.txt is a content guide, not an access control.