What Is llms.txt, and Do You Actually Need One?

Short answer: llms.txt is a proposed standard, a markdown file at yoursite.com/llms.txt, that lists your most important pages so a large language model can find and read them efficiently. It is genuinely useful if you publish developer documentation that AI coding tools consume. It will not get you cited by Google, and no major AI company has publicly committed to reading it in production. Adding one is cheap. Expecting SEO results from it is not reasonable.

This is one of the few topics in search where the marketing has outrun the evidence so far that the honest answer sounds contrarian. Plenty of agencies now sell llms.txt implementation as an AI search deliverable. Google has said, on the record, that it does not support the file. Both of those things are true at once, and only one of them is being repeated in sales decks.

What is llms.txt?

llms.txt is a plain markdown file placed at the root of your domain. It gives a language model a curated map of your site: a short description of what the site is, then a linked list of the pages that matter, sometimes with one-line summaries. A companion file, llms-full.txt, contains the full text of that content in one document.

The problem it was designed to solve is real. Web pages are cluttered with navigation, cookie banners, scripts, and boilerplate. A model reading your documentation wastes most of its context window on markup before it reaches the sentence it needed. A curated markdown file skips all of that. Think of it as robots.txt reversed: rather than telling crawlers where not to go, it tells them exactly where the good material is.

Does Google support llms.txt?

No. Google has stated publicly that it does not support llms.txt and has no plans to, with Gary Illyes confirming this at Search Central Live in 2025. John Mueller has described it as something closer to a temporary crutch for AI tooling than a search feature. Google's own AI features documentation puts it beyond argument: "You don't need to create new machine readable files, AI text files, or markup to appear in these features. There's also no special schema.org structured data that you need to add."

That sentence is worth reading twice before you buy an AI search package built on the opposite claim.

Do ChatGPT and Claude read llms.txt?

There is no public commitment from OpenAI that GPTBot or ChatGPT requests or parses llms.txt during normal browsing, and server-log analyses have generally not shown those crawlers fetching the file. Anthropic publishes llms.txt for its own documentation, which signals interest, but publishing a file is not the same as confirming that Claude consumes it during inference.

Where the file demonstrably works is in developer tooling. AI coding assistants routinely fetch /llms.txt and /llms-full.txt when pointed at a documentation site, because a curated markdown index is exactly what an agent reading an API reference needs. That is the adoption story, and it is a good one. It is simply not a search story.

Who should actually create an llms.txt file?

Type of site	Worth it?	Why
Developer docs, API reference	Yes	AI coding agents fetch it routinely and it measurably improves what they read
SaaS with a technical knowledge base	Probably	Cheap, helps agents and support bots parse your docs accurately
Marketing site or blog	Low priority	No search engine acts on it; your sitemap and content already do this job
Ecommerce store	No	Product data belongs in structured data and a feed, not a markdown index
Local business site	No	Nothing consumes it; effort belongs in content and your Business Profile

The pattern: llms.txt helps when a machine needs to read a body of reference material in full. It does nothing when a machine needs to answer a question and pick a source, which is what search and answer engines are doing.

Will llms.txt help me get cited by AI search?

On current evidence, no. Citations are earned through retrieval. An engine searches, pulls candidate pages, and quotes the ones that answer the question clearly, verifiably, and recently. A file listing your URLs does not make a passage more quotable, and it does not enter that selection process on any major engine.

What does influence citation is unglamorous and well documented: lead with a direct answer under a heading that matches the question as people ask it, be specific enough to be verifiable, use real tables instead of images, show an honest last-updated date, and cover the questions your buyers actually ask. That is the whole of answer engine optimization, and none of it is a file you upload once.

How do you create an llms.txt file?

If you have developer documentation, it takes about fifteen minutes. Create a markdown file at the root of your domain, served at /llms.txt as plain text.

1. Start with an H1 naming the site, then a blockquote summarizing what it does in one or two sentences.
2. Group your key pages under H2 headings such as Documentation, Guides, or API.
3. Under each heading, list links in markdown format, with a short description after each one explaining what the reader will find.
4. Keep it curated. A list of every URL you own defeats the purpose, which is telling a model what is worth reading.
5. Optionally publish llms-full.txt containing the full markdown text of those pages in one file.

Make sure your server returns it as text/plain and that robots.txt does not block it. Then leave it alone. It is a map, not a growth channel, and it needs maintenance only when your documentation structure changes.

Is llms.txt the same as robots.txt?

No, and the resemblance causes real confusion. robots.txt is an established standard that every major crawler obeys, and it controls access by telling crawlers what they may and may not fetch. llms.txt is a proposal that no major engine has committed to, and it controls nothing. It offers guidance about which content is worth reading. One is enforcement, the other is a suggestion that may or may not be heard.

If your goal is to control how AI systems use your content, robots.txt and the standard snippet directives, nosnippet, max-snippet, and data-nosnippet, are the tools that actually work. Be careful with those last three, because restricting snippets also removes you from Google's AI features entirely.

So should you add one?

If you ship developer documentation, yes, this afternoon. It costs almost nothing and the tools that consume it are real and numerous. If you run a marketing site and someone is charging you for llms.txt implementation as an AI visibility service, ask them to show you which engine reads it, and read Google's documentation while you wait for the answer.

The uncomfortable truth is that there is no file, tag, or schema type that makes an AI engine cite you. There is only content that answers a question better than the alternatives, published consistently enough that engines learn to reach for it. That is slower than uploading a text file, which is exactly why so few competitors do it. Rankable researches the questions your buyers ask, writes them answer-first, and publishes on a schedule, so the library that earns citations gets built whether or not anyone remembers to update a markdown index. If AI search is where your traffic went, start with Google AI Overviews optimization and work outward from there. For the documentation itself, note that scanned PDFs and image-only manuals are unreadable to every crawler and model alike, so extract the text and data first if that is where your reference material lives.