Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add PavloSEO/seohead-seotools --skill robots-auditgit clone --depth 1 https://github.com/PavloSEO/seohead-seotoolsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/pavloseo/seohead-seotools/robots-audit)<a href="https://agentmods.dev/skills/pavloseo/seohead-seotools/robots-audit"><img src="https://agentmods.dev/badge/skills/pavloseo/seohead-seotools/robots-audit/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/pavloseo/seohead-seotools/robots-audit"><img src="https://agentmods.dev/badge/skills/pavloseo/seohead-seotools/robots-audit.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00153 | $0.01279 |
| Opus 5 | $0.00077 | $0.00639 |
| Sonnet 5 | $0.00031 | $0.00256 |
| Haiku 4.5 | $0.00015 | $0.00128 |
Grade A, and why
robots-audit scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 9d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
1. **Download it.** `curl -sL -A 'Mozilla/5.0' https://SITE/robots.txt -o /tmp/robots.txt`. How it starts
The opening of the file, as written. The whole thing — 82 lines — stays where its author put it; the contents beside it link to each section on GitHub.
robots-audit
An agentic robots.txt audit built on top of a Screaming Frog crawl. SF shows
"blocked by robots" page by page, but it does not explain WHY a rule is harmful or
HOW to rewrite it. This skill fills that gap.
When to use it
- The user asks to analyze
robots.txt, verify that it does not block important content, or find redundant directives. - An
sf-analyzeraudit reports "Blocked by robots.txt" or unusual index coverage gaps. - A crawl/indexing section is needed before delivering an
sf-reportreport.
Workflow
- Download it.
curl -sL -A 'Mozilla/5.0' https://SITE/robots.txt -o /tmp/robots.txt. Check the HTTP status:curl -sIL https://SITE/robots.txt | head -1. If it is not200, that is already a problem (404 means no rules are present; 5xx may cause crawlers to treat the entire site as blocked). - Parse it by
User-agentgroup. Collect theDisallow/Allow/Sitemap/Crawl-delaydirectives within each group. Remember that rules apply to their ownUser-agentgroup, not globally;*is the fallback. An emptyDisallow:means "allow everything." - Collect reference data. Take the list of live 200-status pages from the
sf-analyzeraudit (audit.json, the internal/200 field) and URLs from the sitemap:curl -sL https://SITE/sitemap.xml | grep -oP '(?<=<loc>)[^<]+'. This represents "what should be indexed." - Checks (heuristics):
- Blocks live, important URLs. For each
Disallow, check whether it matches 200-status pages and/or URLs from the sitemap. A conflict where a page appears in the sitemap but is blocked byDisallowis a red flag. - Blocks rendering resources. A
Disallowfor*.js,*.css,/_next/,/static/,/assets/,/wp-content/, or/wp-includes/breaks rendering, so Google sees an empty page. Severity: high. - Rules are too broad.
Disallow: /*?orDisallow: /*?*blocks EVERY URL with parameters. This often also blocks pagination (/blog?page=2), filters, and UTM landing pages.Disallow: /in the*group blocks the entire site; check whether it was left over from development or staging. - Allow/Disallow conflicts. An
AllowandDisallowin the same group match overlapping paths. State the rule clearly: the longer, more specific matching rule wins, not the rule that appears first in the file. - No
Sitemap:Add the sitemap's absolute URL. Crawl-delayGooglebot ignores it; for most sites it only causes harm by slowing down crawling. Remove it unless there is a real need to throttle an aggressive bot.- Duplicates and typos. Repeated
Disallowdirectives,Dissallow/Disalow,Useragentwithout the hyphen, a relativeSitemap:URL (it must be absolute), and paths without a leading/.
- Blocks live, important URLs. For each
- Create a corrected
robots.txtand show a diff against the original (diff -u /tmp/robots.txt /tmp/robots_fixed.txt). Make minimal changes; do not reformat it merely for the sake of tidiness.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 9d ago First seen · 82 lines · 153 tokens per session scan A f1d45ccc7342
robots-audit is a skill published in the GitHub repository PavloSEO/seohead-seotools (0 stars, last pushed 6d ago), licensed MIT. It adds 153 tokens to every session and 1,279 once invoked, about $0.0008 per session on Opus 5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
fire-your-seo-agency
A procedure for improving how a website appears in search engines and how AI answer systems find and cite it. It covers search, answer-engine, generative-AI, and Naver visibility.
geo-loop
Run one bounded eGEOagents loop iteration over a workspace domain - read the charter and fresh collector data, do ONE unit of work, write substrate artifacts, append one Timeline entry and one LOG line. Use for loop mode, /geo:loop, scheduled GEO runs, or continuous monitoring.
content-scoring
Score content against the 10 GEO criteria with evidence and prioritized fixes. Use when users ask to score, rate, evaluate, or estimate ranking strength.
competitive-analysis
Analyze AI-search competitors for a query and recommend ranking strategy. Use when users ask competitor analysis, who ranks, or competitive landscape.
validation-doctor
Check Brave Search and Chrome DevTools MCP availability and provide exact setup snippets. Use when validation dependencies are missing or uncertain.
geo-content-research
Researches what prompts people ask AI engines (ChatGPT, Gemini, Perplexity, Claude) about a product category and produces a prompts.csv artifact — a prioritized, strictly-schema'd list of the queries where the brand should be cited. Feeds the monitor workflow.