sharozdawa/robotstxt-ai

Visual robots.txt manager for the AI era. Block or allow GPTBot, ClaudeBot, PerplexityBot, and 20+ AI crawlers with one click.

1Stars on the repository

2Mods indexed here, across every type

7mo agoLast push, which is what freshness is scored on

MITLicence, which decides whether bodies are shown

MCP server Claude CodeCodexCursor +2

Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot. Runs locally from the robotstxt-ai-mcp npm package.

not rated 1 7mo ago A tokens not measured original MIT