What is llms.txt? A Complete Guide (+ How to Create & Check It)

If you have looked at your traffic lately, you have probably noticed a new kind of visitor: AI assistants. Tools like ChatGPT, Claude, Perplexity, and Google's AI overviews increasingly read websites to answer questions and cite sources. llms.txt is the emerging standard that helps them read your site correctly — and this guide explains what it is, why it matters, and how to create and check one.

What is llms.txt?

llms.txt is a plain-text markdown file you place at the root of your domain, at https://yoursite.com/llms.txt. It gives large language models (LLMs) a short, curated map of your most important content, written in a format that is easy for AI to parse.

The standard was proposed by Jeremy Howard in 2024. The idea is simple: a normal website is full of navigation, ads, scripts, and styling that waste an AI model's limited context. An llms.txt file strips all of that away and hands the model a clean list of the pages that actually matter, each with a short description.

In other words, llms.txt is to AI what a sitemap is to search engines — but written for humans and models to read, not just machines to crawl.

Why does llms.txt matter?

As more people ask AI assistants instead of typing into a search box, being understood by those assistants becomes its own channel. A clear llms.txt file makes it more likely that an AI will:

• Find your most useful pages instead of guessing
• Summarize your product or docs accurately
• Cite your site as a source in its answers

It will not magically rank you everywhere, but it removes friction. When a model can read a tidy, well-structured file, it spends its context on your actual content instead of fighting through page clutter.

What does an llms.txt file look like?

The format is intentionally minimal. A valid file has:

1. A single H1 title with your site or project name
2. An optional blockquote summary describing the site in a sentence
3. Optional free text for extra context
4. One or more ## sections, each containing a list of markdown links
5. An optional ## Optional section for links that can be skipped to save context

Here is a complete example:

# Example Project

> A short summary of what this site or project is about, in one or two sentences.

Optional extra context for AI systems goes here.

## Docs
- [Getting started](https://example.com/start): how to set things up
- [API reference](https://example.com/api): full endpoint reference

## Optional
- [Changelog](https://example.com/changelog): release history

That is the whole format. Notice how every link has a short description after the colon — that is what tells the model what each page covers.

How to create an llms.txt file

You can write an llms.txt file by hand in any text editor. Here is the process:

1. Start with the title. Add a single # Your Site Name line at the very top.
2. Add a summary. Right below the title, write a > one-sentence summary so AI grasps your site instantly.
3. List your key pages. Create ## Section headings (for example Docs, Guides, Products) and add - [Page name](https://...): description links under each.
4. Mark optional links. Put any "nice to have" links under a final ## Optional section.
5. Save and upload. Save the file as llms.txt and upload it to the root of your domain so it is reachable at https://yoursite.com/llms.txt.

Keep it focused. The point is not to list every page — it is to list the pages you most want an AI to read and cite.

How to check your llms.txt file

Because the format is new, it is easy to make small mistakes: a missing title, a malformed link, or a section with no links. Before you rely on the file, validate it.

The fastest way is to run it through our free llms.txt checker. Paste your file or enter your domain, and the checker validates it against the spec, flags errors and warnings, and gives you a 0–100 health score. If you would rather start from a working file, the tool also loads a valid sample you can edit.

Make a habit of re-running the llms.txt checker whenever your site changes — a docs restructure or a batch of renamed URLs can quietly leave broken links in your file.

llms.txt vs robots.txt: what is the difference?

People often confuse the two because both are root-level text files, but they do opposite jobs:

robots.txt tells crawlers where they may not go. llms.txt invites AI toward the pages you most want it to read. They work side by side — keep your robots.txt for crawl control and add an llms.txt to guide AI.

Common llms.txt mistakes

The most frequent errors are a missing or duplicated H1 title, links that are not written as - [name](url), and sections with no links at all. Skipping the link descriptions is a softer miss — the file still works, but the AI loses the context that makes each page useful. A quick validation pass catches all of these before they matter.

Frequently asked questions

Is llms.txt required? No. It is optional and not yet an official web standard, but adoption is growing fast as AI traffic rises.

Where does the file go? At the root of your domain: https://yoursite.com/llms.txt.

How often should I update it? Whenever the pages you point AI to change. Re-check it each time so it stays accurate.

Does it replace my sitemap? No. Keep your XML sitemap for search engines; llms.txt is an additional, human-readable map for AI.

Get started

llms.txt is one of the simplest, lowest-effort things you can do to make your site AI-ready. Write a short file, list your best pages, and you are most of the way there.

When you are ready, check your llms.txt file to make sure it is valid — and explore our other free tools while you are here.