llms.txt in 2026: What It Is and Whether Your Website Needs One

llms.txt is a proposed Markdown file, placed at the root of a website, that gives language models a short summary of the site and a list of links to its most useful pages. Most business websites do not need one yet: Google says Search ignores it, and no other major AI search engine says it reads one to choose sources.

This article is part of our complete guide: AI Search Optimization.

• Google says you need 0 new AI text files or Markdown versions of pages to appear in Google Search, and that an llms.txt file will neither harm nor help visibility there (Google Search Central's guide to generative AI search).
• Anthropic's crawler page describes 3 bots, all controlled through robots.txt, and does not mention llms.txt (Anthropic's page on how site owners can block its crawlers).
• OpenAI documents 4 crawlers and Perplexity 2, each with robots.txt rules; neither crawler page says its bots read a website's llms.txt to pick sources (OpenAI's crawler documentation).
• Bing's 10 February 2026 post on Copilot citations advises IndexNow and evidenced content, names robots.txt for content preferences, and never mentions llms.txt (Bing Webmaster Blog on AI Performance).

What does an llms.txt file contain?

An llms.txt file contains a title, a one paragraph summary of the website and lists of links to the pages a language model should read first, all written in Markdown.

The proposal, published at llmstxt.org by Jeremy Howard of Answer.AI, sets a simple order. The file opens with a top level heading that names the site or project, which is the only part the proposal requires. A quoted summary follows, then any short notes the owner wants a model to know, then sections with second level headings, each holding a list of links with a brief note on what the linked page covers.

One section name has a special meaning. Links under a heading called Optional are ones a model may skip when it has little room to read, so the most important pages go in the earlier sections. The proposal also suggests offering a clean Markdown copy of each page at the same address with .md added, so a model can read the text without menus, scripts and cookie banners.

Here is what a file could look like for a hypothetical bakery in Leeds. In a real file, each page name and address is written as a Markdown link, with the name in square brackets followed by the address in round brackets; the example below spells them out plainly so they are easy to read. Every detail of the bakery is invented for illustration.

  # Hilltop Bakery, Leeds
  > Hilltop Bakery is a family bakery in Headingley, Leeds, that bakes sourdough and celebration cakes to order. Cake orders need 48 hours of notice. Delivery covers LS postcodes only.
  ## Ordering
  - Celebration cakes: https://www.example.com/cakes.md (sizes, flavours and notice times)
  - Delivery and collection: https://www.example.com/delivery.md (areas, days and collection hours)
  ## Optional
  - Our story: https://www.example.com/about.md (who runs the bakery and since when)

The file is plain text, so anyone can write it in a few minutes with any editor. It goes at the root of the domain, the same place as robots.txt, so the address would be www.example.com/llms.txt. Nothing in it is a rule, a permission or a ranking signal: it is a reading list.

Is llms.txt an official standard?

llms.txt is not an official standard: it is a public proposal from one author, with no standards body behind it and no engine committed to reading it.

That status matters when you compare it with the files search engines do follow. Robots.txt, for example, is described in RFC 9309, an Internet standards track document published by the IETF, and every engine named here documents how its bots treat that file. A proposal can still be useful, and many software projects have adopted llms.txt for their documentation, but adoption by website owners is not the same as use by an AI search engine.

There is one honest sign of where the format has taken hold. OpenAI's own crawler page opens by pointing readers to an llms.txt index of OpenAI's documentation, and Perplexity's documentation site serves one too. Publishing an llms.txt for your own developer documentation is a different act from promising to read everyone else's when choosing what to cite, and neither company's crawler page makes that second promise.

The narrower question of how llms.txt and robots.txt differ, what each one can do and what belongs in each, has its own guide in this series. It sets the two files side by side and shows what to put in each. The short version is that robots.txt asks bots to stay out of parts of a website, while llms.txt only suggests what to read, and cannot keep any bot away from any page.

Do ChatGPT, Google, Perplexity, Claude or Copilot use llms.txt?

None of the five has said it uses llms.txt to choose the pages it cites, and Google has said plainly that Google Search ignores the file.

The table sets out what each company's own documentation says, as checked in October 2026. It covers the five engines most businesses care about, and for each one it names the control that the engine does document. That second column is where your effort is better spent.

Engine	What its documentation says about llms.txt	The control it does document
Google AI Overviews and AI Mode	Google Search ignores llms.txt; it will neither harm nor help	Googlebot access, indexing and snippet rules
ChatGPT search	Crawler page does not mention reading sites' llms.txt files	OAI-SearchBot in robots.txt
Claude	Crawler page does not mention llms.txt	Claude-SearchBot in robots.txt
Perplexity	Crawler page does not mention reading sites' llms.txt files	PerplexityBot in robots.txt
Microsoft Copilot	AI Performance announcement does not mention llms.txt	Bingbot, robots.txt and IndexNow

Absence from documentation is not proof that no system ever opens the file. If you publish one, your server logs may show requests for /llms.txt. A request only shows that something fetched the file, though, not which product it belongs to or whether the file changed any answer.

The engines do describe what decides whether your pages can be used at all. OpenAI says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, Anthropic says blocking Claude-SearchBot may reduce visibility in search results, and Perplexity says PerplexityBot surfaces and links websites in its results. Every one of those statements is about robots.txt and bot access, which is why that file comes first.

Can an llms.txt file help or harm your website?

An llms.txt file is unlikely to help or harm your AI search visibility; the realistic risk is a file that slowly goes out of date.

Google's guide to generative AI search says it is completely fine to create and maintain llms.txt files, and that doing so will neither harm nor help a site's visibility or rankings in Google Search. The same guide lists creating unnecessary AI text files among the tactics to ignore. That leaves no case for llms.txt as a Google ranking project, and no reason to fear one either.

The practical risks are small but real. A file written once and never updated can point to pages that have moved, prices that have changed or services you no longer offer, and any reader that does use it gets old facts. If you add Markdown copies of pages, you also have two versions of every page to keep in step, which is more work than most business owners expect.

The file also cannot protect anything. Leaving a page out of llms.txt does not stop any bot from reading it, and listing a page does not invite a bot that robots.txt or a firewall turns away. Owners sometimes add an llms.txt hoping to signal that AI engines are welcome, while a security setting elsewhere refuses those same bots, and the file changes nothing about that.

When is an llms.txt file worth adding?

An llms.txt file is worth adding when your website holds a lot of reference material, such as software documentation, an API guide or a large help centre.

The format was designed for that job. A developer who asks a coding assistant about your product benefits when the assistant can find a tidy index of your documentation and read clean Markdown copies of each page. Software companies, publishers of technical manuals and organisations with large knowledge bases have a real reason to try it, because their readers often paste documentation into AI tools themselves.

For most businesses the case is weaker. A plumber in Nairobi with eight pages, or a hypothetical accountant in Leeds with a services page and a contact page, gains little from a reading list of a website that is already short and easy to crawl. If your website platform publishes an llms.txt with one click, there is no harm in switching it on, but it should not take an hour away from the work below.

There is a sensible middle path for owners who want one anyway. Treat the file as a summary of the website, not as a marketing page. Keep it short, list only the pages you would want quoted, make every summary match the page it describes, and put a reminder in your calendar to check it whenever prices, services or opening hours change.

What matters more than llms.txt for AI citations?

Bot access, a clear answer at the top of each important page and mentions on other websites matter far more, because the engines document them.

Start with access, because nothing else counts if the search bots cannot read your pages. Check that robots.txt lets in OAI-SearchBot, Claude-SearchBot, PerplexityBot, Googlebot and Bingbot, and that no firewall, CDN or security plugin turns them away at the server. OpenAI says a robots.txt change can take about 24 hours to reach its systems, and Perplexity says up to 24 hours, so an access fix shows quickly.

Then look at the pages themselves. Each important page should open with a direct answer to the question it targets, back that answer with specific facts and say who the business is and where it works. Google's guide makes the same point in its own terms: a page must be indexed and eligible to show with a snippet to appear in its AI features, and it needs no special files for that.

The quickest way to see where you stand is to scan your website with the AI Search Visibility Checker. It checks robots.txt for each engine's search bot, asks for your homepage as each bot to catch firewalls, and gives each page an AI Citation Score out of 100 with the passage an AI would quote. It reports whether you have an llms.txt file but does not score it, since no engine has said the file affects citations, and the scoring method explains each weighting.

How do you add an llms.txt file if you decide to?

You add an llms.txt file by writing a short Markdown file in the proposal's order and uploading it to the root of your domain, so it opens at yourdomain/llms.txt.

Begin with the title line and a two or three sentence summary that a stranger could repeat accurately: what the business does, where, and for whom. Then group your most useful pages under a few plain headings, with one line on what each covers, and put background pages under Optional. Use full addresses, because the file may be read away from your website.

Check the file the way a reader would. Open the address in a browser and confirm it shows as plain text rather than a download or an error page, then click through each address to make sure none is broken or redirected. If your firewall challenges unknown visitors, check that the file is reachable without a browser too, since a challenge page would replace it for any bot.

Finally, make it part of your routine rather than a one off task. A reading list is only as good as the pages it points to. Update it on the same day you change a service page, keep the summary in line with your homepage and Business Profile, and remove it if you find you cannot keep it current, because an accurate website with no llms.txt beats an inaccurate file.

Questions business owners ask about llms.txt

Will an llms.txt file get my website into ChatGPT answers?

No public statement from OpenAI says so. OpenAI's crawler documentation ties inclusion in ChatGPT search answers to OAI-SearchBot and robots.txt, and does not mention reading websites' llms.txt files. Allowing OAI-SearchBot and writing pages with clear answers is the documented route.

Does Google use llms.txt for AI Overviews?

No, Google says Google Search ignores llms.txt files. Its guide states that creating one will neither harm nor help a site's visibility or rankings in Google Search, which includes AI Overviews and AI Mode. Google asks only that a page be indexed and eligible to show with a snippet.

Can llms.txt block AI training bots?

No, llms.txt cannot block any bot. To ask a training bot such as GPTBot, ClaudeBot or Google-Extended to stay away, you add a group for it in robots.txt with a Disallow rule. A firewall or CDN setting is the way to enforce a block, since robots.txt is a request that well behaved bots honour.

Does llms.txt replace an XML sitemap?

No, the two do different jobs. An XML sitemap lists the addresses you want search engines to crawl and index, and Google and Bing read it. llms.txt is a short, hand picked reading list with summaries, and no search engine has said it uses one in place of a sitemap.

Should I remove an llms.txt file my website already has?

Not if it is accurate, because Google says the file does no harm. Check that each address works and each summary matches its page. Remove it only if it is out of date and you cannot keep it current.

Your next step on llms.txt

Spend the next hour on the documented work, and give llms.txt five minutes at most.

Run a scan to confirm every AI search bot can reach your homepage and that your five most important pages open with a clear answer. Fix any blocked bot first, because that is the change OpenAI and Perplexity say takes effect within about a day. Then rewrite the opening paragraph of any important page that lacks a direct answer.

If you then want an llms.txt, write a short and accurate one using the bakery example as a pattern, list only pages you would be happy to see quoted, and add a monthly reminder to review it. Keep a note of the date you published it, so you know how old it is. Revisit the decision whenever an engine publishes a clear statement about the file, since the engines' own documentation is the only reliable guide to what they read.