llms-txt-audit

llms.txt Audit

Audit a website's /llms.txt file and produce an actionable report for AI/LLM discoverability.

When to use this skill

Use this skill when the user asks to:

• Audit, validate, or check an llms.txt or llms-full.txt file
• Make a site discoverable and usable by AI assistants, chatbots, and LLMs
• Find broken, redirecting, or blocked links inside llms.txt
• Confirm llms.txt follows the llmstxt.org structure (H1 title, summary, link sections)
• Decide what should go into llms.txt vs. llms-full.txt

Background: what llms.txt is

llms.txt is a proposed standard (https://llmstxt.org) for a markdown file at the site root (/llms.txt) that gives LLMs a concise, curated map of the site's most useful content. Unlike robots.txt (which controls crawling) or sitemap.xml (which lists every URL), llms.txt is hand-curated and human-readable. Its structure:

# Project Name              ← required: the only mandatory element

> A short summary.          ← recommended blockquote summary

Optional free-form markdown with key context (no headings).

## Docs                     ← H2 section of curated links
- [Quickstart](https://example.com/quickstart): How to get started.
- [API reference](https://example.com/api): Full API docs.

## Optional                 ← an "Optional" section may be skipped by LLMs with limited context
- [Changelog](https://example.com/changelog): Release history.

A companion /llms-full.txt may contain the full expanded content in one file. This skill audits structure and link health against that spec.

Inputs to collect

Before starting, confirm with the user:

1. Site URL or llms.txt URL — a live site root (e.g., https://example.com, which fetches /llms.txt) or a specific llms.txt URL (--llms).
2. Scope — how many referenced links to probe (default 300, capped with --max-urls).
3. Probing — whether to fetch each link to check status/redirect/noindex (default on). Use --no-probe for a fast structure-only pass.