Getting it into your agent
This one installs as part of its plugin. Adding the marketplace and installing the plugin brings it with everything else the plugin ships.
/plugin marketplace add hex/claude-crawl/plugin install claude-crawlWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/hex/claude-crawl/web)<a href="https://agentmods.dev/skills/hex/claude-crawl/web"><img src="https://agentmods.dev/badge/skills/hex/claude-crawl/web/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/hex/claude-crawl/web"><img src="https://agentmods.dev/badge/skills/hex/claude-crawl/web.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00150 | $0.02107 |
| Opus 5.5 | $0.00060 | $0.00843 |
| Sonnet 5.5 | $0.00030 | $0.00421 |
| Haiku 4.5 | $0.00015 | $0.00211 |
Grade A, and why
web scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 173 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Web Content Operations
All web operations use shell scripts in ${CLAUDE_PLUGIN_ROOT}/scripts/. Output goes to
.claude-crawl/ in the project root to protect the context window from large content.
Before any operation: mkdir -p .claude-crawl/{reads,searches,crawls,extractions,reranks,ground,screenshots,pdfs}
WebFetch calls are automatically intercepted and routed through Jina Reader for cleaner
markdown output. Set CLAUDE_CRAWL_NO_INTERCEPT=1 to disable this behavior and use
WebFetch directly. Run check-auth.sh --verbose to see which services are available.
Routing Table
Choose the right tool based on what you have and what you need:
HAVE NEED USE
─────────────────────────────────────────────────────────────────────
Topic/question Search results jina-search.sh or fc-search.sh
URL, static page Clean markdown jina-read.sh
URL, JS-heavy/SPA Rendered content fc-scrape.sh
URL, need visual capture Screenshot or PDF cf-page.sh --format screenshot|pdf
URL, need structured data JSON extraction (AI) cf-page.sh --format json --ai-prompt "..."
Domain All URLs on site fc-map.sh
Multiple URLs, bulk Multi-page content cf-crawl-start.sh or fc-crawl.sh
Statement/claim Fact verification jina-ground.sh
Multiple results Ranked by relevance jina-rerank.sh
How to decide between similar tools
Search: Use jina-search.sh for general web search (fast, returns full content). Use
fc-search.sh when you need Firecrawl-specific features like --scrape (auto-scrapes each
result) or want an alternative search provider.
Single page fetch: Start with jina-read.sh — it's fastest and cheapest. If the result
is thin (< 100 words, no headings — the script warns you), escalate to fc-scrape.sh which
renders JavaScript. If you need Cloudflare's AI extraction or screenshots, use cf-page.sh.
What ships with it
3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 173 lines · 150 tokens per session scan A 85776aab3f3b
web is a skill published in the GitHub repository hex/claude-crawl (1 stars, last pushed 7mo ago), licensed MIT. It adds 150 tokens to every session and 2,107 once invoked, about $0.0006 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
test-site
Tests a deployed, activated Power Pages site at runtime using browser-based navigation, page crawling, and API request verification via Playwright. Use when the user wants to test, verify, or smoke-test their deployed site.
preview-screens
Use when the user wants to preview generated screens in a browser without starting Metro / a simulator — for example after /create-mobile-app finishes or after /edit-app regenerates a screen.
qa
Browser-based QA verification. Launches a real browser, navigates the app, clicks buttons, fills forms, and tests user flows. Works as a standalone skill or as a phase end condition in campaigns. Requires Playwright (optional dependency, graceful skip if not installed).
broken-link-checker
Scans a website to find broken links (404s, 500s). Crawls internal pages, identifies broken outbound links, and reports source pages for easy fixing. Use this when the user asks to "check for broken links", "find 404s", "audit my links", or "is my site healthy".
responsiveness-check
Test website responsiveness across viewport widths using browser automation. Resizes a single session through breakpoints, screenshots each width, and detects layout transitions (column changes, nav switches, overflow). Produces comparison reports showing exactly where layouts break. Trigger with 'responsiveness…
Dead Link Detector
Automated detection and reporting of broken links, missing resources, and orphaned pages across web applications using crawl-based and DOM-based strategies.