Does llms.txt matter? What the evidence says in 2026

Probably not, if the goal is being found or quoted by search engines and AI assistants. llms.txt is a Markdown file that lists a site's key pages for AI agents. Google's documentation says Search ignores it, and none of the crawler pages from OpenAI, Anthropic or Perplexity mentions reading it. Whether to publish one depends on who will keep it accurate.

What is llms.txt, and who proposed it?

llms.txt is a plain Markdown file, served at the root of a site or at a subpath, that gives AI agents a short guide to what the site contains. Jeremy Howard published the proposal (opens in a new tab)^[1] on 3 September 2024. The current page, labelled version 2, was last modified on 10 August 2026.

The format has a fixed order. An H1 with the site or project name is the only required part. After it come an optional blockquote summary, optional plain text without headings, and any number of H2 sections holding lists of links. Each list item is a Markdown link, optionally followed by a colon and a note. By convention an "Optional" section holds secondary links an agent can skip.

It is a content guide, not a set of crawling rules. The spec says llms.txt information is used "on demand, when an agent needs information about a topic while assisting a user". It adds that the files are "used most heavily for software documentation", where coding agents follow them. Neither llms.txt nor robots.txt authenticates visitors or protects private content: RFC 9309 (opens in a new tab)^[17] says robots.txt rules are "not a form of access authorization", and a login does that job.

What does Google say about llms.txt?

Google has answered directly. Its Search documentation updates log, in an entry dated 15 June 2026 (opens in a new tab)^[2], says these files "aren't needed for Google Search (and won't negatively or positively impact your visibility or rankings)". It adds that keeping them for other services is fine. The note was added to Google's guide to generative AI features (opens in a new tab)^[3], which lists llms.txt among the things to ignore and says Google Search "ignores them". A separate AI features page (opens in a new tab)^[4] says: "You don't need to create new machine readable files, AI text files, or markup to appear in these features."

Those statements cover Google Search, including AI Overviews and AI Mode. They say nothing about other companies' systems.

Other Google teams run other products. Chrome's Lighthouse documentation for its agentic browsing audits (opens in a new tab)^[5], last updated 5 May 2026, calls llms.txt "an emerging convention" and says that without the file, agents "may spend more time crawling the site". The audit flags a page if retrieving the file causes a server error, and a 404 is marked not applicable "as providing the file is optional at the moment". Google's own Gemini API documentation (opens in a new tab)^[6] serves an llms.txt as well.

Do OpenAI, Anthropic or Perplexity say they read it?

No public statement found. Their crawler pages describe user agents and robots.txt controls, and none mentions crawlers reading llms.txt.

What these pages do say concerns robots.txt. OpenAI recommends allowing OAI-SearchBot in robots.txt, and requests from its published IP ranges, for a site to appear in ChatGPT search results. Perplexity gives the same advice for PerplexityBot. Anthropic says disabling Claude-SearchBot "may reduce your site's visibility and accuracy in user search results".

Each company serves an llms.txt for its own developer documentation, as the table links show, and the OpenAI and Perplexity crawler pages carry a banner pointing readers to theirs. That is a choice about its own documentation, not a statement that its search products consult the file on yours. Meta's crawler documentation (opens in a new tab)^[13], updated 21 May 2026, does not mention it either. "No public statement found" covers only these pages and is not proof that no product reads the file.

What do server logs and studies show?

Three vendor studies bear on the question. They measure different things, so their figures are not combined here.

Both log studies find few AI-crawler requests for the file. In EZY's logs, the GPTBot family and ClaudeBot fetched robots.txt thousands of times and llms.txt fewer than ten times each. Those are EZY's labels, and they say nothing about OpenAI's or Anthropic's search or assistant products. In Ahrefs' sample during May 2026, no AI-bot requests appeared among the 404s for files that did not exist, which does not show that no AI system ever checks for the file. Among named AI bots, Ahrefs found GPTBot, a training crawler, fetching the files most and Claude Code, a coding agent, second, ahead of every AI search and assistant bot. It calls coding agents "the closest thing to an intended audience in our data", which fits the proposal's focus on documentation. EZY found Meta's crawler fetching llms.txt more often than robots.txt, which Meta's documentation does not describe.

A fetch is not use: Ahrefs calls its figures "a ceiling on actual llms.txt consumption". SE Ranking tried to predict how often domains are cited in LLM answers from several factors, one of them whether the domain had a file, and its predictions improved when that factor was left out, which it reads as the file adding noise rather than signal. It compared existing sites, and its page omits the time window, the models sampled and how files were detected. EZY says its panel over-represents small businesses that were shopping for an AI-visibility product, and EZY offers a tool that generates llms.txt files. SE Ranking's 10.13% and Ahrefs' 28% come from different populations, so they show no trend. None of the three tests what happens when a site adds a file.

What does publishing one cost, and what might it return?

A decision rule follows from the table. If your site can generate the file from data it already maintains, as Plandora's does from its page list and article list, upkeep is automatic and the remaining risk is a wrong source list. If someone must edit the file by hand, publish it only when a named person will recheck every link whenever a page moves, retires or changes, and otherwise skip it. Neither case is a visibility tactic, because none of the sources above shows a visibility benefit today.

What matters more than llms.txt?

Five recommended checks come from Google's and the operators' documentation. They are checks, not demonstrated effects: the sources do not rank them or measure a citation benefit from any of them. The first three concern whether crawlers can find and read the page, and the last two concern readers.

1. Let the right crawlers in. Allow the search crawlers you want, such as Googlebot, OAI-SearchBot, Claude-SearchBot and PerplexityBot, in robots.txt, and check that firewall rules do not block them. Google's guide says to ensure content is crawlable, and OpenAI and Perplexity recommend that site owners permit their published IP ranges. How AI crawlers read your website covers the checks.
2. Put the substance in the page text. Google can process JavaScript content that is not blocked, but its guide says SEO on JavaScript frameworks is "generally more complex". Keep the main text in the page's HTML rather than behind scripts.
3. Keep sitemaps accurate. Google says a sitemap (opens in a new tab)^[19] tells search engines which pages you think are important and can carry each page's last-updated date, though it does not guarantee crawling. List only pages you want found, and keep the dates truthful.
4. Answer first, under clear headings. Google says "You don't need to write in a specific way just for generative AI search", so the reason is the reader. Its guide says readers appreciate pages organised by paragraphs, sections and headings, and that there is no requirement to break content into tiny pieces for AI. Answer-first writing for people and AI shows how.
5. Show who wrote it and when. Google's guidance on byline dates (opens in a new tab)^[18] asks for a prominent visible date, labelled with text such as "Last updated", matched by datePublished and dateModified in structured data. Put a named author on the page too.

The wider picture is in generative engine optimisation for expert businesses.

What goes in an llms.txt file?

Example only: a fictional pottery studio with placeholder addresses.

# Example Pottery Studio
> A fictional studio used to show the format.

Classes run in six-week blocks.

## Classes
- [Beginner wheel-throwing](https://example.com/classes/beginner): six-week course, no experience needed

## Optional
- [Studio history](https://example.com/about/history)

The H1 with the site name is the only required line. The blockquote gives an agent arriving cold a short summary, and any plain text below it carries more detail, with no headings. Each H2 starts a list of links, where every item is a link followed by an optional colon and note. The "Optional" section holds secondary links an agent can skip when it needs less context. The proposal defines only llms.txt, a short index whose detail lives behind its links. Plandora's site code also generates a separate llms-full.txt that holds every article as Markdown in one file. That file is a Plandora choice, not part of the proposal.

To publish one:

1. Pick the pages. Link only to pages you maintain and would want quoted, grouped under headings that match how visitors think.
2. Write plain notes. Give each link one factual line. Ahrefs advises keeping the file to links and descriptions, with nothing instruction-shaped.
3. Point to it. Ahrefs concluded that AI tools fetch the file when a link, an index or a user tells them it exists, so reference it from your pages or documentation.
4. Test and maintain it. The spec suggests asking an agent questions about your content, "giving it only your llms.txt as a starting point". The agent then has to follow your links, so a question it cannot answer can point to a broken or thin linked page. Keep the file in version control, limit who can edit it and review it when pages change.

Does Plandora publish one?

Yes. plandora.com serves an llms.txt that is generated from the site's list of public pages and its published guides, so a new guide appears in the file without hand-editing. The site code also generates the llms-full.txt described above. Plandora's own editorial standard is not to present either file as a ranking lever, and the evidence above gives no reason to.