26 tagged crawler, measured the same way as everything else here.
Browse within: browser-automation 12bun 12cdp 12curl-impersonate 12audit 7core-web-vitals 7data-extraction 7
Agent Claude Code
Documentation specialist for CrawlForge MCP Server APIs and tools. Creates comprehensive documentation, usage examples, and integration guides. Use when adding new features or updating APIs.
Agent Claude Code
Performance monitoring and optimization specialist. Tracks system performance, identifies bottlenecks, and ensures optimal resource usage. Use during load testing and optimization phases.
Agent Claude Code
Project manager for CrawlForge MCP Server development. Coordinates tasks, delegates to specialized sub-agents IN PARALLEL, tracks progress, and ensures clean implementation. Use PROACTIVELY for any multi-step project coordination.
Agent Claude Code
Use when a Bxc scraper fails, gets blocked (403/429/captcha), or returns empty/wrong data. Inspects HTML / headers / console dumps and proposes the next profile escalation step.
Agent Claude Code
Use proactively for any task needing fresh web data — answering a question from live sources, multi-source research, scraping a page/site, extracting structured data, monitoring a SERP, or checking what a URL actually contains. Drives the bxc CLI (search/scrape/recon/detect/mirror) end-to-end and returns a sourced…
Agent Claude Code
Use proactively when Rust bridge (rust-bridge/, src/rust/), Zig DOM bridge (src/zig-bridge/), or generic FFI loader code is added or modified. Audits memory safety, async correctness, lifetime invariants, and the Bun native boundary.
Agent
Focus on page-level copy and extractable text. Treat word counts, title length, meta description length, repeated exact phrases, long paragraphs, and placeholder text as deterministic heuristics. Do not judge expertise or truthfulness without evidence.
Agent
Focus on extractability and AI-search readiness heuristics: direct answer blocks, semantic headings, FAQ-like areas, structured data, author or organization signals, dates, citations, and autonomous passages.
Agent
Focus on objective crawl and page-structure signals: status, redirects, canonical, robots, HTTPS, lang, viewport, H1 count, heading depth, sampled internal links, and accessible link names. Prefer hard evidence from the shared context.