scrape-le
01MCP server Claude CodeCodexCursor
Analyse robots.txt content and report whether a path may be crawled. Runs locally from the scrape-le-mcp npm package.
1 8d ago A
tokens not measured
original MIT
1Stars on the repository
7Mods indexed here, across every type
8d agoLast push, which is what freshness is scored on
MITLicence, which decides whether bodies are shown