agentmods.dev

coolaigit/site-crawler-mcp

Site-wide crawler MCP server: BFS crawl of all internal links with time/title/URL filters, page operations (summarize/click/download), SQLite persistence. Built on crawl4ai.

Open on GitHub
0Stars on the repository
1Mods indexed here, across every type
16d agoLast push, which is what freshness is scored on
MITLicence, which decides whether bodies are shown
Sort RelevanceRecently updatedFewest tokensSecurity gradeRepo starsStars this weekTop rated 1 mods

site-crawler-mcp

01

coolaigit/site-crawler-mcp

MCP server Claude CodeCodexCursor +2

MCP server "site-crawler-mcp" as configured in coolaigit/site-crawler-mcp. Runs locally from the site-crawler-mcp Python package.

not rated 0 16d ago A tokens not measured original MIT