llms.txt is a Markdown file at the site root that tells AI clients what the site is and where its important pages are. It is not a ranking signal; it is a convenience file that makes a site easier to summarise.
Expectations should be set correctly. Google stated in its May 2026 AI optimisation guidance that llms.txt is not needed for AI Overviews or AI Mode, and OpenAI does not describe it as a ranking signal. An SE Ranking scan of 300,000 domains put adoption at roughly 10%.
Publishing it is still cheap, and some agents do read it. To make it useful: use absolute URLs, list only the languages your site genuinely serves well, and generate it automatically — a hand-maintained llms.txt goes stale quickly.
What belongs in the file?
The format is defined at llmstxt.org and it is Markdown: an H1 with the site name, a one-sentence description below it, then sections of links. A short note beside each link is what makes the file do its actual job.
Three practical rules: use absolute URLs (relative paths fail to resolve in some clients), list only languages whose content is genuinely in good shape, and generate the file automatically. A hand-maintained llms.txt starts contradicting your sitemap within months.
Setting the right expectation
Treat any source claiming llms.txt is a ranking signal with scepticism. In its May 2026 guidance on AI features, Google stated plainly that the file is not required for AI Overviews or AI Mode. OpenAI does not describe it as a ranking signal either.
Adoption is limited too: SE Ranking's scan of 300,000 domains put it at roughly 10%. Publishing the file is still cheap — it takes minutes to generate and some agents do read it. But your priority should be allowing the right bots in robots.txt, not llms.txt.
Frequently asked
Can I appear in AI search without llms.txt?
Yes. The file is not a precondition. What is a precondition is that the page can be crawled and indexed — which means the relevant agents are allowed in robots.txt.
How does llms.txt differ from robots.txt?
robots.txt is an access rule that crawlers enforce. llms.txt is an introduction file; it neither blocks nor opens anything. For visibility, robots.txt is the file that decides.
Do I need llms-full.txt?
Not necessarily. It concatenates full content into one file, which stops being practical on large sites as the file grows. Get llms.txt right first.
