llmstxt-guide

llms.txt Guidelines

Author and maintain a project's /llms.txt file (and the per-page markdown mirrors it points to), so LLMs can ingest the site at inference time within a bounded context window.

What llms.txt Is

A markdown file served at /llms.txt (or a subpath) that gives an LLM a curated, concise map of a site's most useful content. Complements: does not replace: robots.txt (which is about crawler permissions) and sitemap.xml (which is exhaustive and HTML-oriented).

File Format (strict ordering)

The file must contain these elements in this order:

1. H1 heading (required): project or site name
2. Blockquote (optional): short summary with key understanding info
3. Descriptive content (optional): zero or more markdown sections, excluding headings
4. H2-delimited file lists (optional, repeatable): curated link sections

Only the H1 is required. The other sections are optional but follow this ordering when present.

Link list grammar (inside an H2 section)

## Section name

- [Link title](https://example.com/page.md): Optional notes
- [Another link](https://example.com/other.md)

Each list item is:

• A markdown hyperlink such as [name](https://example.com/page) (required)
• Optionally followed by : and brief notes

The "Optional" section

An H2 section literally titled ## Optional designates secondary resources that can be skipped when a shorter context is needed. Processors use this distinction to produce two variants of expanded context.

Markdown Mirror Convention

For every HTML page worth giving to LLMs, serve a clean markdown version at the same URL with .md appended:

• https://example.com/docs/intro.html → https://example.com/docs/intro.html.md
• https://example.com/docs/intro → https://example.com/docs/intro.md
• https://example.com/docs/intro/ → https://example.com/docs/intro/index.html.md
• https://example.com/ → https://example.com/index.html.md

The llms.txt file links to these .md mirrors, not the HTML originals.