What is llms.txt, and does it actually help?

An llms.txt file is a plain Markdown file on your website that gives AI agents a short, curated guide to your most useful pages. It is a proposal, published by Jeremy Howard on September 3, 2024, and not a ratified web standard. The proposal's own site describes it as "a proposal to standardise on using an /llms.txt file to provide information to help agents use a website."

Does it help? As of October 2026, nobody can show you first-party evidence that it improves your visibility in AI answers. Google has said clearly that Google Search ignores these files, including for its AI features. We found no published statement from OpenAI, Anthropic, Perplexity or Microsoft, either way, about whether their search products read llms.txt on other people's websites.

The honest verdict is short: it costs little to create, the benefit is unproven, and it should sit well below the basics on your to-do list.

What the proposal says

The current text at llmstxt.org is titled "The /llms.txt file, v2." The author describes it as "v2 of the proposal, updated based on what I learned from two years of adoption." It is still described as "open for community input," with a GitHub repository hosting an informal overview.

The idea has two halves.

The index file

The file is written in Markdown with sections in a set order:

• An H1 with the name of the project or site. This is the only required section.
• A blockquote with a short summary.
• Optional plain Markdown sections with more detail (no headings).
• Zero or more sections under H2 headings, each containing a list of links. Every entry is a Markdown link, optionally followed by a colon and a note about the page.

By convention, an H2 section named "Optional" holds secondary links an agent can skip when it needs a shorter context.

The file can sit at the site root or at any path within it, covering the pages under that path. Where more than one file applies, the proposal says agents should use the most specific one.

Markdown versions of pages

The second half asks sites to provide a clean Markdown version of a page at the same URL with .md appended or with the extension replaced by .md. Version 2 adds discovery links: a rel="alternate" type="text/markdown" link pointing to the Markdown version of a page, and a rel="describedby" link pointing to the llms.txt file that covers it.

What about llms-full.txt?

You will see llms-full.txt mentioned in many guides. The current proposal text does not mention it at all. It exists as a convention some vendors use on their own documentation sites. Anthropic's developer documentation, for example, serves both files.

What it was designed to do

The proposal is specific about its purpose.

• On-demand use by agents. "Agents are expected to view or search llms.txt to find the information they need, then follow the relevant links."
• Mainly inference, not training. "Our expectation was that llms.txt would mainly be useful for inference rather than training, and that is how it has been used, though training runs could take advantage of the information too."
• Mostly documentation. "llms.txt files are used most heavily for software documentation, where coding agents follow them to find API references and tutorials."

The proposal says the same structure can serve "a business outlining its structure and policies," but the use it reports is documentation. A local service business is a long way from that.

The proposal also separates the file from the two files you already know. A robots.txt file "lets automated tools know what access to a site is considered acceptable." And "while sitemaps list all pages for search engines, llms.txt offers a curated overview for LLMs."

Who has said what

Here is what each company has published, as of October 2026.

Google's position, in its own words

Google's guide to optimizing for generative AI features lists "LLMS.txt files and other 'special' markup" under things you don't need to do: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them."

It then adds: "It's completely fine if you decide to create and maintain LLMS.txt files (or other similar files) for other services or systems that use these files. Doing so will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them."

Google also notes that it "may discover, crawl, and index many kinds of files in addition to HTML on a website: this doesn't mean that the file is treated in a special way." Google added a clarifying note on this topic to its documentation on June 15, 2026. Our article on Google AI Mode covers what Google does ask for.

The Lighthouse audit is a different thing

Chrome's Lighthouse tool, which developers use to test pages, has an llms.txt audit in its agentic browsing category. It calls the file "an emerging convention." If your site has no file, the audit is marked Not Applicable. It only flags a problem if a server error occurs when it tries to retrieve the file. Both statements are true at once: Google Search ignores the file, and a Chrome developer tool checks for it as an optional item. Neither one tells you it improves visibility.

The other AI companies

The proposal's author points out that OpenAI, Anthropic and Gemini publish llms.txt files for their own developer documentation. Perplexity's documentation does too. That is the documentation use case again: they publish a file so that coding agents can read their docs. None of those pages says the company's crawler or consumer search product reads llms.txt on your website. Their crawler documentation lists robots.txt as the control for site owners and says nothing about reading llms.txt on your site.

So two popular claims are both unsupported. "ChatGPT reads your llms.txt" has no first-party source. "No AI engine reads llms.txt" has no first-party source either. The only company we found on record is Google, and its answer for Google Search is no.

What llms.txt cannot do

• It does not block or allow crawlers. It is not an access control. To manage which AI bots can fetch your pages, use robots.txt. See our guide to AI crawlers and robots.txt.
• It does not opt you out of AI training. The vendors that offer a training opt-out do it through a named user-agent token in robots.txt.
• It does not replace a sitemap. A sitemap lists all pages for search engines. This file is a curated subset.
• It does not fix weak pages. The file only links to your content. If the pages are thin, vague or out of date, a tidy index of them changes nothing.

How to create one

If you still want the file, keep the effort proportional. There are two routes.

Let your platform generate it

Sources: Yoast's llms.txt feature page, Rank Math's knowledge base and Wix's help center. For other platforms, check the vendor's own documentation before assuming the feature exists. We have a step-by-step walkthrough for setting up llms.txt with Yoast, and a broader guide to GEO for WordPress sites.

Rank Math's own documentation is candid about the limits. It says the file's "adoption across platforms is still limited."

Write it by hand

1. Create a plain text file named llms.txt.
2. Start with one H1 line: your business or site name.
3. Add a blockquote with a one- or two-sentence summary of what you do and for whom.
4. Add H2 sections for your main content groups, such as services, pricing, guides and company information.
5. Under each, list your best pages as Markdown links, with a short note after a colon. The proposal says links should point to LLM-friendly content, such as Markdown versions of pages, where you have them.
6. Upload the file to your site root so it loads at yourdomain.com/llms.txt.
7. Review it whenever you add, rename or remove a page it links to.

A hypothetical example for a plumbing company:

# Example Plumbing Co.

> Licensed residential plumber serving Springfield. Repairs, water heaters and drain cleaning.

## Services
- [Water heater repair](https://example.com/water-heater-repair/): what we fix, brands serviced, how pricing works
- [Drain cleaning](https://example.com/drain-cleaning/): methods and what to expect on the visit

## Company
- [Service area](https://example.com/service-area/): neighborhoods and zip codes covered
- [Contact](https://example.com/contact/): phone, hours and booking

Keep it short; listing every URL is a sitemap's job. To test it, the proposal's author suggests "asking an agent questions about your content, giving it only your llms.txt as a starting point." That shows the file is usable when handed to an agent. It does not show that any AI search product fetches it unprompted.

The honest verdict

Low cost and unproven benefit is a reasonable basis for a five-minute task. It is a poor basis for a line item in a proposal. The work with documented effects is less exciting: pages that are crawlable and indexed, content available as text, accurate business data, and crawler access set up on purpose. Our technical foundations of GEO guide covers that list, and Microsoft's documented tools are covered in Bing Webmaster Tools and IndexNow for AI search.

Conclusion

An llms.txt file is a tidy idea with a real use in software documentation and no confirmed effect on how AI search products treat a business website. Google has said its Search ignores the file; we found no statement from the other major AI companies about reading it on third-party sites. Create one if it's a switch in your CMS, keep it accurate, and spend your effort on the pages it points to. If you want to know which fixes would matter for your site, request a free GEO audit. More guides are in our GEO and AEO Guide.

Frequently asked questions