aeo-visibility

AEO — being citable, not just rankable

Classic SEO gets you ranked. AEO gets you quoted. The mechanics differ: answer engines extract self-contained claims from crawlable text, and they weight consistency of facts about an entity across the whole web.

1. Let the crawlers in — deliberately

• siteConfig.aiCrawlers decides this — { search, userFetch, training } — and src/app/robots.ts writes a named group per crawler:
  • search (OAI-SearchBot, Claude-SearchBot, PerplexityBot) — index for AI answers and cite. Block them and the site leaves those answers.
  • userFetch (ChatGPT-User, Claude-User, Perplexity-User) — a person asked an assistant to open the page.
  • training (GPTBot, ClaudeBot, CCBot, Google-Extended, Applebot-Extended, Meta-ExternalAgent, Bytespider) — model training. Google-Extended never affects Google Search.
• Ask the user before changing it. "Be cited by AI" and "don't train on my content" are different goals: blocking training while allowing search is a coherent position; blocking everything and expecting citations is not.
• Check the CDN / WAF too. Many hosts' bot protection blocks AI crawlers without anyone deciding to — curl -A "OAI-SearchBot/1.0" -I <url>/ must answer 200 on the deployed host.
• Verify server-side rendering: answer-engine crawlers are far less reliable at executing JavaScript than Googlebot. Anything client-rendered may be invisible to them. In this starter, content is server-rendered and animations only animate — keep it that way.
• AI crawlers get the robot form. src/utils/bot-ua.ts lists them next to the search engines, so the proxy serves them the page at rest (no intro, every line of copy in the HTML). The starter's original list knew search engines and lab tools but not AI crawlers — they got the people form, where streamed copy sat in hidden segments. When a new AI crawler appears, add it there (one regex for the proxy and isBot()).
• Read the page as they do — no JS: curl -s -A "GPTBot/1.0" <url>/ (and ClaudeBot, PerplexityBot). The <h1> and body copy must be present and not under a hidden ancestor. On production sites built from the old starter an empty app/loading.tsx put the whole page inside <div hidden>; after deleting it, two sites went from h1 HIDDEN and 394 / 1,619 hidden characters to 0. Text a client-only scene paints into a canvas doesn't exist for them — keep an HTML twin.