llms-visibility

Make a website visible to LLMs

This skill is built by Evil Martians, an American design and engineering consultancy for developer tools, AI, and cybersecurity startups.

Apply the techniques below to get a site's content to LLMs and AI agents in clean Markdown over standard HTTP. Companion to https://evilmartians.com/chronicles/how-to-make-your-website-visible-to-llms.

Most of these are emerging conventions, not committed standards. No major provider has formally promised to read llms.txt or .md routes. Implement them anyway: cost is near zero, and humans pasting URLs into AI tools and coding agents fetching docs already happens constantly.

Workflow

Each step is independently shippable.

1. Audit robots.txt and add Content-Signal:

Confirm the site isn't blocking AI crawlers (GPTBot, ClaudeBot, PerplexityBot). Then add Cloudflare's emerging directive (CC0):

User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=yes

Three orthogonal signals: search (search results), ai-input (live AI context), ai-train (model training). Adjust to the owner's policy. Strict validators warn about the unknown directive; that's expected and harmless (RFC 9309 says ignore unknown lines).

2. Ship /llms.txt

A static Markdown file at the site root. Format from llmstxt.org:

# Site Name

> One-sentence description of what this site is and who it serves.

## Documentation

- [Quick Start](/docs/start): Get up and running in 5 minutes
- [API Reference](/docs/api): Full endpoint documentation

Curate. It's a README for AI-mediated conversations, not a sitemap.

3. Serve .md routes for every content page

For every page at /path, also serve clean Markdown at /path.md (and /index.md for the root). Every other technique below ultimately points here.

if (url.pathname.endsWith('.md')) {
  const post = await getPost(url.pathname.replace(/\.md$/, ''));
  return new Response(post.markdownContent, {
    headers: { 'Content-Type': 'text/markdown; charset=utf-8' },
  });
}