kopachlager/prerenderbuddy-mcp
MCP server Claude CodeCodexCursor
Local MCP tools for crawler-readability and discovery-file diagnostics. Runs locally from the @prerenderbuddy/mcp npm package.
4 tagged gptbot, measured the same way as everything else here.
kopachlager/prerenderbuddy-mcp
MCP server Claude CodeCodexCursor
Local MCP tools for crawler-readability and discovery-file diagnostics. Runs locally from the @prerenderbuddy/mcp npm package.
MCP server Claude CodeCodexCursor
Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot. Runs locally from the robotstxt-ai-mcp npm package.
MCP server Claude CodeCodexCursor
Four free tools to check if AI engines can find, read and recognise a site. No API key. Runs locally from the crawlbit-mcp npm package. Needs 1 environment variable to run.
taylorsmithgg/crawl-census-client
MCP server Claude CodeCodexCursor
Ask before you fetch: will this domain serve your crawler, refuse it, or charge it? Remote server at crawlcensus.com.