Crawlsonar
AI readiness & GEO2 min readUpdated 2026-07-09

What is llms.txt and how to write one

llms.txt is a simple Markdown file that points AI systems at your most important content. Here's what it is, what goes in it, and how to publish a valid one.

llms.txt is a proposed standard: a single Markdown file at the root of your site (/llms.txt) that gives large language models a clean, curated map of your most important pages — the way sitemap.xml does for search engines and robots.txt does for crawl rules.

Note: It's an emerging convention, not yet universally consumed. It's cheap to add and signals intent — but it complements good HTML and structured data, it doesn't replace them.

Why publish one#

  • It hands AI systems a curated shortlist instead of making them guess from your whole site.
  • It uses Markdown — the format LLMs read most reliably — with links and short descriptions.
  • It's a clear, machine-readable statement of what you consider your canonical, citable content.

The format#

A valid llms.txt starts with an H1 (your site name), an optional blockquote summary, then H2 sections of links with short descriptions:

# Acme Widgets

> Durable widgets for industrial use, shipped worldwide.

## Docs

- [Getting started](https://acme.com/docs/start): install and first widget in 5 minutes
- [API reference](https://acme.com/docs/api): full endpoint reference

## About

- [Company](https://acme.com/about): who we are and where we ship

Use absolute URLs. Keep descriptions to one line. A companion llms-full.txt can contain the full text of key pages for models that want the content inline.

Publish and validate#

  1. Save the file as llms.txt and serve it at https://yourdomain.com/llms.txt with content type text/plain or text/markdown.
  2. Link only to pages that are public, canonical and server-rendered.
  3. Validate structure and link health with the llms.txt Checker, or generate a first draft with the llms.txt Generator.

Frequently asked questions

▸ Where does llms.txt go?

At the root of your domain: https://yourdomain.com/llms.txt — the same location pattern as robots.txt and sitemap.xml.

▸ Is llms.txt the same as robots.txt?

No. robots.txt sets crawl permissions (what bots may access). llms.txt is a curated content map for LLMs — closer in spirit to a sitemap, but written in Markdown with descriptions.

▸ Do AI companies actually read llms.txt?

Adoption is still early and inconsistent. Treat it as a low-cost, forward-looking signal rather than a guaranteed input. Good HTML, structured data and crawler access matter more today.

Do it with a free tool

Related guides

Published 2026-07-09 · Updated 2026-07-09 · By Crawlsonar.