AI search visibility
What is llms.txt (and do you need one?)
Last researched October 27, 2026. Facts may change. Re-check the sources before you act.
If you sell online or publish product reviews, you have probably seen people say you need an llms.txt so ChatGPT and other AI tools can find you. This guide explains what the file actually is, how to write one, where it helps, and where the evidence is still thin.
The short answer
llms.txt is a proposed Markdown file, usually published at https://yoursite.com/llms.txt, that gives AI agents a curated map of your site: a short description plus links to the pages (ideally clean Markdown versions) that matter most.
It was proposed by Jeremy Howard / Answer.AI in September 2024 and updated to v2 in August 2026 (llmstxt.org). It is not an official standard from OpenAI, Google, or Anthropic. It is an open proposal that many sites have adopted, especially documentation platforms.
Do you need one? It is cheap to add, and it is useful for coding agents and for people or tools that deliberately open the file. There is no solid public evidence yet that adding llms.txt alone increases how often ChatGPT or similar systems cite your brand. Treat it as infrastructure, like a sitemap, not a ranking hack.
Why the file exists
Web pages are built for humans: navigation, ads, scripts, and layout. Agents that pull a page into a context window spend tokens on that chrome, and a whole site is usually too large to load at once.
llms.txt tries to solve that by giving agents:
- A short overview of what the site is.
- A curated list of links to the most useful LLM-friendly pages.
- Optional pointers to Markdown versions of those pages, which are cleaner than HTML.
Jeremy Howard's original write-up frames it as authors knowing best what an LLM should read (Answer.AI, 3 Sep 2024). The v2 update (10 Aug 2026) adds discoverability via standard link relations and clarifies that a file can live in a subpath, for example /docs/llms.txt, and covers pages under that path (Changes – llmstxt.org).
How it differs from robots.txt and sitemap.xml
| File | Job |
|---|---|
| robots.txt | Tells crawlers what they may fetch. Access rules. |
| sitemap.xml | Lists URLs for discovery and indexing. Often huge. |
| llms.txt | A curated, human-and-LLM-readable map of what matters, with short notes. |
They complement each other. OpenAI's publisher guidance still emphasizes allowing crawlers such as OAI-SearchBot in robots.txt for search features. It does not say llms.txt drives citations (OpenAI Help – Publishers and developers FAQ). Re-check the current wording. Google's public AI Search guidance has likewise focused on existing Search systems and signals, not on llms.txt as a ranking input. Industry coverage of that point includes the Search Engine Journal summary of the SE Ranking analysis.
The format (v2)
Per llmstxt.org, a conforming file uses Markdown in this order:
- An optional byte-order mark.
- An H1 with the site or project name. This is the only required section.
- An optional blockquote summary.
- Optional detail paragraphs, with no headings.
- Zero or more H2 sections that are file lists: Markdown bullet links, with optional notes after a colon.
- By convention, an Optional H2 for secondary links that agents can skip when context is tight.
v2 also recommends:
- Clean Markdown mirrors of important pages, such as
page.html.mdorpage.md. - Discoverability with
rel="alternate"andtype="text/markdown"for a Markdown twin, andrel="describedby"for the covering llms.txt, as HTML link tags or HTTP Link headers. - The most specific file wins when several llms.txt files could apply (the root, versus
/docs/, versus a deeper path).
Docs platforms such as Mintlify and GitBook, and some CMSs, can generate the file. The integrations list is on llmstxt.org.
A simple example you can copy
Here is a minimal file for a small review site. Adapt the names and URLs. Do not invent pages you do not have. The domain below is an example, not a real site.
# Acme Tools Reviews
> Independent reviews of ecommerce and AI visibility tools for small online stores. Prices dated on each page. No invented star ratings.
We publish comparison tables with public pricing and a last-checked date. Affiliate links are disclosed on every page that has them.
## Start here
- [Home](https://example.com/): What we cover and how we pick tools.
- [Methodology](https://example.com/methodology): What we will and will not claim.
- [Affiliate disclosure](https://example.com/disclosure): How we make money.
## Guides
- [What is llms.txt](https://example.com/guides/what-is-llms-txt.md): This guide in Markdown.
- [How to get cited by ChatGPT](https://example.com/guides/chatgpt-citations.md): Practical checklist (research-based).
## Optional
- [Full text summary](https://example.com/llms-full.txt): Longer plain-text dump of key pages.
- [Sitemap](https://example.com/sitemap-index.xml)
Keep the language concise. Describe each link in one short clause. Avoid unexplained jargon. Then test by giving an agent only your llms.txt and asking what the site covers.
Webflow's walkthrough below explains the same proposal from a documentation-platform angle. The studies in the next section are the evidence this page uses. The video is not a CartCited test.
Does it actually help AI citations?
Be honest with yourself, and with readers.
What large studies reported (as of late 2025 through 2026):
- SE Ranking analyzed roughly 300,000 domains. About 10.13% had an llms.txt. Statistical tests and an XGBoost model found no clear link between having the file and how often a domain was cited in major LLM answers. Removing the feature even improved the model's accuracy (Search Engine Journal summary; SE Ranking's own write-up).
- Ahrefs looked at about 137,000 sites in their analytics. 28% published llms.txt, but 97% of those files got zero fetches in the May 2026 window they studied. Most of the rare traffic was bots. AI search bots barely used the files, and no AI bot requested a missing file. They do not go looking. Coding agents, in the Ahrefs breakdown, were relatively more interested than AI-search crawlers (Ahrefs: We Analyzed 137K Sites).
- A smaller before-and-after look, Search Engine Land's 10-site check, also failed to show a clean causal lift from the file alone (Does llms.txt matter?).
Practical takeaway: If the goal is to be cited by ChatGPT, prioritize crawl access in robots.txt, clear entity information, original useful pages, and mentions on other sites. An llms.txt is still reasonable to add because it costs almost nothing and helps when an agent is pointed at your site, especially for docs and product structure. It is not a substitute for good content.
CartCited has not run its own controlled citation experiment for this article. We haven't tested this hands-on yet, so this page does not claim that adding the file changed anyone's citations.
Semrush's video asks the same citation question from the vendor's side. The studies above are what this page relies on. The video is extra context, not a result we measured.
Who should add one (and who can skip)
Worth doing if you:
- Publish documentation, API references, or a product knowledge base.
- Want coding agents or support bots to navigate the site with less scraping noise.
- Already maintain Markdown versions of key pages, or your docs platform generates them.
- Run a content or affiliate site and want a tidy start-here map for agents, and for yourself.
Lower priority if you:
- Have a single landing page with almost no content depth.
- Expect the file alone to move AI Overview or ChatGPT rankings.
- Cannot keep the links accurate. A stale or wrong map can mislead agents. Ahrefs also flags security and prompt-injection probing of public llms.txt files. Keep yours boring and accurate.
How to publish one in practice
- Write the Markdown using the format above.
- Host it at
/llms.txton the site root, with a plain-text or Markdown content type. Static hosts, including Cloudflare Pages, Netlify, GitHub Pages, and Vercel, can serve a file frompublic/llms.txtor the equivalent. - Link important pages to Markdown twins where you can, and optionally add
rel="alternate"andrel="describedby". - Keep robots.txt open for the AI crawlers you want, and block the ones you do not. llms.txt does not replace that file.
- Re-test periodically. Open your
/llms.txtin a browser, then ask an agent with browsing to use only that file to answer what the site covers.
A live example is this site's own file: /llms.txt. A later post will cover how this site's agent-readable files are maintained. That post is not published yet.
FAQ
Is llms.txt required by OpenAI or Google?
No. It is a community proposal. Major platforms have not announced it as a required ranking signal.
Is it the same as robots.txt?
No. robots.txt is about permission to crawl. llms.txt is about what to read first.
Should links point to HTML or Markdown?
The proposal prefers LLM-friendly Markdown versions when they exist. HTML links still work. Markdown is cleaner for agents.
Can I put llms.txt only under /docs/?
Yes, in v2. A file covers URLs under its path. The most specific applicable file should win.
Will this get me cited by ChatGPT?
There is no reliable public evidence that the file alone does that today. Focus on content quality, crawlability, and external visibility first.
Do AI crawlers always fetch it?
No. In the Ahrefs May 2026 sample, most published files were never fetched. Coding agents look more interested than AI-search bots when fetches do happen.
Is there a risk to publishing one?
Keep it accurate. Treat it like a public map of your best pages, not a dump of private admin URLs.
Bottom line
Add an llms.txt if you can maintain it. Follow the v2 Markdown format, link your best pages, preferably as Markdown, and pair the file with a sane robots.txt. Do not expect an overnight jump in AI citations. Use the file as a clear front door for agents, then spend the real effort on original pages that people and models both have a reason to quote.
Sources
Checked October 27, 2026.