Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add lionkiii/claude-seo-skills --skill seo-robots-aigit clone --depth 1 https://github.com/lionkiii/claude-seo-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/lionkiii/claude-seo-skills/seo-robots-ai)<a href="https://agentmods.dev/skills/lionkiii/claude-seo-skills/seo-robots-ai"><img src="https://agentmods.dev/badge/skills/lionkiii/claude-seo-skills/seo-robots-ai/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/lionkiii/claude-seo-skills/seo-robots-ai"><img src="https://agentmods.dev/badge/skills/lionkiii/claude-seo-skills/seo-robots-ai.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00063 | $0.01702 |
| Opus 5 | $0.00032 | $0.00851 |
| Sonnet 5 | $0.00013 | $0.00340 |
| Haiku 4.5 | $0.00006 | $0.00170 |
Grade A, and why
seo-robots-ai scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 12d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 204 lines — stays where its author put it; the contents beside it link to each section on GitHub.
AI Crawler Robots.txt Audit
Analyzes a site's robots.txt specifically for AI crawler access policies.
Complements /seo-technical (which does a broad robots.txt check) with
deep AI-specific analysis.
@skills/seo/references/ai-crawlers-guide.md
AI Crawler Registry
Training crawlers (respect robots.txt, used for model training)
| Bot Name | Owner | Purpose |
|---|---|---|
| GPTBot | OpenAI | Training data collection |
| ClaudeBot | Anthropic | Training data collection |
| Google-Extended | Robots.txt control for Gemini / AI training (not Search) | |
| Applebot-Extended | Apple | Apple Intelligence training |
| CCBot | Common Crawl | Open dataset used by many AI models |
| Bytespider | ByteDance | TikTok / AI training |
| cohere-ai | Cohere | AI model training |
| FacebookBot | Meta | Meta AI training |
| Meta-ExternalAgent | Meta | Meta AI training crawler |
| Amazonbot | Amazon | Alexa / AI training |
| Diffbot | Diffbot | AI knowledge graph |
| ImagesiftBot | ImagesiftBot | AI image training |
| Omgili | Webz.io | AI data feeds |
User-triggered / search crawlers (fetch on behalf of a live user or search index)
Blocking these removes you from AI search answers and citations.
| Bot Name | Owner | Purpose |
|---|---|---|
| OAI-SearchBot | OpenAI | ChatGPT search index (not training) |
| ChatGPT-User | OpenAI | ChatGPT browsing (real-time, user-triggered) |
| PerplexityBot | Perplexity | AI search engine index |
| Perplexity-User | Perplexity | User-triggered fetch for Perplexity answers |
| anthropic-ai | Anthropic | Claude web fetch |
| Google-CloudVertexBot | Vertex AI Agents — crawls on site owners' request | |
| DuckAssistBot | DuckDuckGo | DuckAssist AI answers |
| MistralAI-User | Mistral | User-triggered fetch for Le Chat citations |
| Meta-ExternalFetcher | Meta | User-initiated Meta AI fetches |
Important: Blocking training bots ≠ blocking AI search visibility. Blocking training crawlers only opts you out of model training. Blocking user-triggered / search crawlers removes your site from AI answers and citations (ChatGPT search, Perplexity, DuckAssist, etc.).
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 12d ago First seen · 204 lines · 63 tokens per session scan A b88732074351
seo-robots-ai is a skill published in the GitHub repository lionkiii/claude-seo-skills (20 stars, last pushed 3mo ago), licensed MIT. It adds 63 tokens to every session and 1,702 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
seo-images
Image SEO audit for a URL or domain. Pulls raw image inventory via Firecrawl, then audits alt-text quality, modern-format coverage (WebP / AVIF), responsive sizing (srcset / sizes), lazy-loading and LCP signals (loading, fetchpriority, decoding), CLS-safe dimensions, descriptive file names, and ImageObject JSON-LD.…
seo-technical-audit
Focused one-shot technical SEO audit for a domain. Crawlability, indexability, security, mobile, structured data, JS rendering — single-pass deliverable, not a diff. Distinct from seo-drift (which tracks changes over time) and from seo-page (which audits keywords/traffic for one URL, not technical health). Use when…
seo-api
DataForSEO API integration architect for developers. Covers the entire DataForSEO surface — DataForSEO Labs (ranked keywords, keyword ideas/suggestions/related, domain rank overview, competitors, domain/page intersection, subdomains, relevant pages, bulk keyword difficulty, bulk traffic estimation, search intent…
seo-ads
Paid-search competitive landscape for a domain or keyword. Reconstructs the paid footprint keyword-by-keyword from live SERP ad blocks — advertisers, ad copy patterns, who else bids on the same keywords, CPC/competition signals, SERP shopping/ad-pack visibility — and produces a competitive ads brief plus a recommended…
seo-backlinks-profile
Full backlink profile for a domain — referring domains, anchor text distribution, authority distribution, IP and subnet diversity, growth/decay trend, toxic-candidate flagging. Distinct from seo-backlink-gap (which is gap-vs-competitor only). Produces a profile health score and reviewable disavow candidate list (never…
seo-content-audit
E-E-A-T + CITE quality audit for an EXISTING piece of content. Scores Experience, Expertise, Authoritativeness, Trustworthiness, and citation-readiness for AI search; surfaces veto items that block publication; produces a publish / publish-with-fixes / no-publish verdict. Distinct from seo-content-brief (produces a…