llms.txt Checker: is your llms.txt file valid and working?

What llms.txt is

llms.txt is a plain-text Markdown file at the root of a website that gives large language models a short, curated map of the site: what the site is, and which pages matter most, with a one-line note on each. Jeremy Howard of Answer.AI proposed it in September 2024, and the format is documented at llmstxt.org. The idea is simple. A model with a limited context window shouldn't have to wade through navigation, scripts and cookie banners to work out which of your pages answers a question. You hand it the list instead.

It is a proposal, not a web standard like robots.txt, and it does not control access. Anything that can read your pages can still read them whether or not the file exists.

The format, in one example

# Example Co

> Example Co makes invoicing software for freelancers and small agencies in the US and UK.

Plans start at $12/month. All features are available on every plan.

## Docs
- [Getting started](https://example.com/docs/start): create an account and send a first invoice
- [Recurring invoices](https://example.com/docs/recurring): schedules, reminders and late fees

## Company
- [Pricing](https://example.com/pricing): plans, limits and billing FAQ
- [Contact](https://example.com/contact): support hours and sales email

## Optional
- [Changelog](https://example.com/changelog): release notes since 2022

The parts, in order:

1. An H1 with the site or project name. This is the only required section.
2. A blockquote summary. One or two sentences with the facts a model needs to understand everything below it.
3. Optional plain paragraphs or lists with extra context. No headings here.
4. H2 sections of links. Each item is a Markdown link, optionally followed by a colon and a short note: - [Title](URL): note.
5. An ## Optional section, if you want one. Its links can be skipped when a tool needs a shorter context.

What this checker tests

• Is the file there? We request /llms.txt and report the HTTP status and any redirect. A 404 means there is no file.
• Is it really text? Many sites answer every unknown URL with their homepage or a "page not found" template and a 200 status. We detect an HTML page served at /llms.txt and flag it, because a model that fetches it gets your homepage markup, not a map.
• Does it follow the format? H1 on the first line, a single H1, a blockquote summary, H2 sections, list items that are real Markdown links, notes on those links, and full https:// URLs rather than relative paths.
• Do the links work? We request the first 10 unique links and report each status. A curated list that points at redirects and 404s is worse than no list.
• Size and type. The Content-Type header, the byte size and a rough token estimate (bytes divided by four).
• llms-full.txt. Whether the companion file with full page text exists.

How to read your results

Red items break the file for anyone who reads it: no H1, sections with no links, or dead links. Amber items won't stop a model from reading the file but make it less useful: no summary, bare list items, links without notes, relative URLs, a wrong content type. Grey items are information only.

The most common failure we see is not a formatting problem. The site has no llms.txt at all, but the server returns the homepage with a 200 status, so a quick check in a browser looks fine. This checker reports that as "HTML instead of text".

How to write a good llms.txt

• Curate, don't dump. List the 10 to 40 pages that answer the questions people actually ask about you: product and pricing pages, key guides, policies, contact details. Your sitemap already lists everything.
• Write the summary for a stranger. Who you are, what you sell or publish, who it is for, and where you operate, in plain words.
• Make every note specific. "Pricing: plans, limits and refund policy" helps a model choose. "Click here" does not.
• Link to the final URL. No redirects, no tracking parameters, no pages behind a login.
• Keep it current. When you remove or move a page, update the file. Our link test catches the ones you forget.

To publish it, upload a file named llms.txt to your web root so it loads at https://yourdomain.com/llms.txt. On WordPress that is the same folder as wp-config.php; some SEO plugins can also generate one for you.

Will llms.txt help you get cited by AI search?

Be realistic about this. The file is cheap to make and it helps tools that do read it, such as coding assistants and agents that fetch documentation. But the large AI search products have not documented llms.txt as a signal for choosing which sources to cite, and Google has said publicly that Google Search does not use it. What still decides citations is whether AI crawlers can reach your pages (test that with our AI Crawler Checker), whether the content renders without JavaScript, and whether each page answers a question clearly. Read what llms.txt is and whether you need one and how to get your website cited by ChatGPT for the bigger picture.

Frequently asked questions

Is llms.txt the same as robots.txt?

No. robots.txt tells crawlers what they may fetch. llms.txt is a reading list: it points models at your most useful pages and blocks nothing. To keep AI crawlers out of a page, use robots.txt or your server settings.

What is llms-full.txt?

An optional companion file that contains the full text of the pages listed in llms.txt, in one Markdown document, so a tool can load all of it in a single request. It suits documentation sites. Most business sites only need llms.txt.

Where does the file have to live?

At the root path /llms.txt. The proposal allows it in a subpath too, but tools look at the root first, so that is the one this checker tests.

Can a bad llms.txt hurt my SEO?

Search engines don't rank pages on it, so a broken file won't cost you rankings. It will mislead any tool that reads it, which is why dead links and an HTML page served at /llms.txt are worth fixing.