Where alternate files give AI the clean content of a single page, llms.txt describes your whole site. It is one markdown file at your site root that gives AI systems a curated map: what your site is, and which pages are worth reading. Think of it as a short guided tour written for a machine, not a list of every URL you have.
Who reads it, honestly #
llms.txt is a convention, not a standard that every AI company has agreed to. Anthropic has confirmed support and Perplexity has said it retrieves llms.txt to help decide which pages to look at. Coding agents such as Cursor and the Claude desktop app read it. Independent server-log studies suggest most published llms.txt files are rarely requested, so it is worth being clear-eyed: this is a low-cost file that helps in specific situations, not a switch that changes how often AI mentions you.
It is most useful when an AI system arrives at your domain without prior context and has to work out what you do and where to look. It costs a few minutes to write and nothing to host, which is why it is worth doing even while adoption is still settling.
How it differs from a sitemap #
A sitemap.xml is a complete list of URLs built for crawlers to discover pages. It says where your pages are, not what they contain, and it aims for coverage. llms.txt is the opposite: it is short, curated, and written to be read. It leaves most of your site out on purpose.
| Sitemap.xml | llms.txt | |
|---|---|---|
| Audience | Search-engine crawlers | AI systems and agents |
| Format | XML list of URLs | Markdown, readable by a person |
| Goal | Complete coverage | Curated selection |
| Typical size | Every page you have | 5 to 20 pages |
| Contains | URLs and last-modified dates | Site summary plus described links |
The format #
The file is plain markdown and the structure is fixed. A single H1 with your site name, a blockquote summary, then H2 sections containing lists of links. Each link is a markdown link, optionally followed by a colon and a description.
# Acme Robotics
> Industrial pick-and-place arms for warehouse automation.
> Last reviewed 2026-08-14.
## Products
- [AR-7 Arm](https://acmerobotics.com/products/ar-7): 7-axis arm, 12kg payload, 1.4m reach. Spec sheet and CAD downloads.
- [Gripper Kits](https://acmerobotics.com/products/grippers): Vacuum and two-finger end effectors in four sizes.
## Documentation
- [Install Guide](https://acmerobotics.com/docs/install): Mounting, power and network setup for AR-series arms.
- [ROS 2 Driver](https://acmerobotics.com/docs/ros2): Package install, topics and services.
## Optional
- [Company History](https://acmerobotics.com/about/history)- The H1 is the only required element. Use exactly one, and put it first.
- The blockquote directly under the title is your summary. This is the line an AI is most likely to reuse when describing your company, so make it accurate rather than promotional.
- Every list item must be a real markdown link. A bare URL on its own line is the single most common mistake and it makes the file unreadable to a parser.
- A section named exactly "Optional" is reserved by the convention. It means these links can be skipped when a shorter answer is needed.
What makes one worth reading #
The descriptions are the part that does the work. An AI deciding which of your pages to read decides on the description, not the URL. "Product page" tells it nothing. "7-axis arm, 12kg payload, spec sheet and CAD downloads" tells it exactly when this page is the right answer.
- Include 5 to 20 links. Past about 50 the file stops being a curated map and each entry gets diluted.
- Keep the whole file under about 10kB so it stays cheap to read.
- Write specifically. If a description could be pasted onto any other company's site, it is not earning its place.
- Skip marketing language. Taglines and metaphors rely on context an AI does not have.
- Date the file. A line such as "Last reviewed 2026-08-14" tells a reader whether to trust it.
- Group links under headings that mean something: Products, Documentation, Pricing, Support.
Common mistakes #
- Bare URLs instead of markdown links. The file will not parse as llms.txt.
- More than one H1, or an H1 that is not the first line.
- Dumping your whole sitemap into it. Volume is the opposite of what this file is for.
- Links that have moved or 404. Stale links make a reader distrust everything else in the file.
- Outdated product names or prices. An llms.txt with wrong information is worse than none, because the wrong information travels.
Where to put it #
The file belongs at the root of your domain, served as plain text or markdown at https://yourdomain.com/llms.txt. On most platforms that means dropping it in the folder that serves static files: public/ for Next.js, Nuxt and Vite, static/ for Hugo and SvelteKit, the site root for WordPress, and the Files area for Webflow or Squarespace. Confirm it worked by visiting the URL and checking you get the raw text back rather than a styled page or a 404.