🕷️ A lightweight Model Context Protocol (MCP) server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents. Similar to Firecrawl's API but self-hosted and free. Perfect for integrating web scraping into your AI workflows with OpenAI Agents SDK, Cursor, Claude Code, and other MCP-compatible tools.
2 files for Cursor and Claude Code: cursorrules, crawl4ai-mcp-server CLAUDE.md — 5,754 tokens loaded in every session.
CLAUDE.md A 4,140 tok .cursorrules A 1,614 tok These files are sadiuysal/crawl4ai-mcp-server's own configuration — they tell Cursor and Claude Code how to work on this repository, so they are not mods to install elsewhere. Copy one as a starting point and replace the parts that are about this project.