24 tagged crawler, measured the same way as everything else here.
Browse within: cursor 5data-extraction 5
MCP server Claude CodeCodexCursor +2
Scrapling is an undetectable, powerful, flexible, high-performance Python library that makes Web Scraping easy and effortless as it should be! Runs locally from the scrapling Python package.
MCP server Claude CodeCodexCursor +2
MCP server "webreaper" as configured in alex-on-ai/WebReaper. Launched with WebReaper.Mcp.
MCP server Claude CodeCodexCursor +2
Search the web and extract clean, readable text from webpages. Process multiple URLs at once to sp… Remote server at server.smithery.ai.
MCP server Claude CodeCodexCursor +2
Technical SEO in Claude: GSC + crawl + DataForSEO - audit, market sizing, content recon. Runs locally from the @houtini/seo-audit-console npm package. Needs 9 environment variables to run.
MCP server Claude CodeCodexCursor +2
MCP server "deepspider" as configured in ma-pony/deepspider. Runs src/mcp/server.js with node.
MCP server Claude CodeCodexCursor +2
Crawl and analyse websites for SEO errors using Crawlee with SQLite storage. Runs locally from the @houtini/seo-crawler-mcp npm package. Needs 2 environment variables to run.
serkan-ozal/driflyte-mcp-server
MCP server Claude CodeCodexCursor +2
One of 2 in server.json
Discover available topics and explore up-to-date, topic-tagged web content. Search to surface the… Remote server at server.smithery.ai.
MCP server Claude CodeCodexCursor +2
MCP server "lookacrawler" as configured in lucasmartins-ai/lookacrawler. Runs locally from the lookacrawler npm package.
MCP server Gemini CLI
One of 3 in settings.json
Gives the agent a persistent knowledge graph it can write facts to and recall across sessions. Runs locally from the @modelcontextprotocol/server-memory npm package.
MCP server Claude CodeCodexCursor +2
Web scraping, crawling, deep research & autonomous extraction — 31 MCP tools, clean Markdown/JSON. Runs locally from the crawlforge-mcp-server npm package. Needs 3 environment variables to run.
MCP server Claude CodeCodexCursor +2
Universal web scraper with LLM-ready markdown, RAG chunking, PDF/DOCX support. Runs locally from the thecrawler npm package.
kopachlager/prerenderbuddy-mcp
MCP server Claude CodeCodexCursor +2
Local MCP tools for crawler-readability and discovery-file diagnostics. Runs locally from the @prerenderbuddy/mcp npm package.
MCP server Claude CodeCodexCursor +2
MCP server "crawmcp" as configured in bilginerdem/craw-mcp. Runs locally from the crawmcp npm package.
MCP server Claude CodeCodexCursor +2
MCP server for the Synoppy web-data API — read, screenshot, crawl, map, search, extract, classify, enrich, and images as tools for any MCP client. Runs locally from the @synoppy/mcp npm package.
maxaeo/maxaeo-ai-visibility-mcp
MCP server Claude CodeCodexCursor +2
Local-first MCP server for AI visibility, GEO/AEO, llms.txt, and AI crawler readiness audits. Runs locally from the maxaeo-ai-visibility-mcp npm package.
MCP server Claude CodeCodexCursor +2
Native Codex SEO CLI for technical audits, performance, Search Console, baselines, historical trends, and CI quality gates. Runs locally from the codex-seo npm package.
MCP server Claude CodeCodexCursor +2
Eval-backed discovery for the auxiliar.ai gateway — the best web-access provider per job, measured. Runs locally from the auxiliar-mcp npm package.
MCP server Claude CodeCodexCursor +2
Model Context Protocol server for web crawling powered by Crawl4AI. Runs locally from the mcp-crawl4ai Python package.
MCP server Claude CodeCodexCursor +2
Give any LLM agent the power to read any website — even the ones that block bots. An MCP server that returns clean Markdown and handles blocks automatically via residential proxies. Runs locally from the hatfetch npm package.
MCP server Claude CodeCodexCursor +2
Fast HTML/XML Template Compiler. Runs locally from the chameleon Python package.
MCP server Claude CodeCodexCursor +2
MCP server "site-crawler-mcp" as configured in coolaigit/site-crawler-mcp. Runs locally from the site-crawler-mcp Python package.