crawl4ai

crawl4ai is an MCP server for Claude Code, Codex, Cursor, Gemini CLI, OpenCode from walksoda/crawl-mcp. Its token cost is not measured, scanned A, original, MIT.

An MCP server for crawling websites with Crawl4AI, a tool that fetches and processes web pages for an AI assistant. It is configured to run as a Python module and can use several language-model providers.

In plain words
What is it for?
Running the Crawl4AI MCP service and connecting it to an AI client for web-crawling tasks.
Why use it?
It provides a configured way for an MCP client to access website-crawling functionality, subject to the required environment setup and API credentials.

MCP server for Claude CodeCodexCursorGemini CLIOpenCode

Written for no agent in particular: it speaks MCP, so any client can run it. Also seen: listed in a Claude config.

Not installable: its command points at a path on the author’s own machine, so it runs nowhere else. The line is /home/user/prj/crawl.

Good fit Running the Crawl4AI MCP service and connecting it to an AI client for web-crawling tasks.

Compare 6 mcp servers from other repositories ↓
Install

Getting it into your agent

There is no command for this one: it runs only inside a plugin, and the catalogue could not identify which plugin ships it. The source is linked below.

Made for: Claude Code, Codex, Cursor, Gemini CLI, OpenCode.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for crawl4ai

README.md
[![agentmods](https://agentmods.dev/badge/mcp/walksoda/crawl-mcp/crawl4ai/github.svg)](https://agentmods.dev/mcp/walksoda/crawl-mcp/crawl4ai)
Your own site
<a href="https://agentmods.dev/mcp/walksoda/crawl-mcp/crawl4ai"><img src="https://agentmods.dev/badge/mcp/walksoda/crawl-mcp/crawl4ai/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for crawl4ai

Your own site · 80×15
<a href="https://agentmods.dev/mcp/walksoda/crawl-mcp/crawl4ai"><img src="https://agentmods.dev/badge/mcp/walksoda/crawl-mcp/crawl4ai.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session not measured What this adds to a session before it is invoked.
When invoked not measured Not applicable: nothing here is loaded into a session.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Security

Grade A, and why

crawl4ai scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 9d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

configs/claude_desktop_config.json · 70 lines

What it actually says

{
  "crawl4ai": {
    "command": "python",
    "args": [
      "-m",
      "crawl4ai_mcp.server"
    ],
    "cwd": "/home/user/prj/crawl",
    "env": {
      "PYTHONPATH": "/home/user/prj/crawl/venv/lib/python3.11/site-packages"
    },
    "llm_config": {
      "default_provider": "openai",
      "default_model": "gpt-4.1",
      "providers": {
        "openai": {
          "api_key": null,
          "api_key_env": "OPENAI_API_KEY",
          "base_url": null,
          "models": [
            "gpt-4.1",
            "gpt-4.1-mini",
            "gpt-4.1-nano",
            "gpt-4o",
            "gpt-4o-mini",
            "gpt-4-turbo"
          ]
        },
        "anthropic": {
          "api_key": null,
          "api_key_env": "ANTHROPIC_API_KEY",
          "base_url": null,
          "models": [
            "claude-3-5-sonnet-20241022",
            "claude-3-haiku-20240307"
          ]
        },
        "ollama": {
          "api_key": null,
          "api_key_env": null,
          "base_url": "http://localhost:11434",
          "models": [
            "llama3.3",
            "qwen2.5"
          ]
        },
        "aoai": {
          "api_key": null,
          "api_key_env": "AZURE_OPENAI_API_KEY",
          "base_url": null,
          "base_url_env": "AZURE_OPENAI_ENDPOINT",
          "api_version": "2025-04-01-preview",
          "models": [
            "gpt-4.1",
            "gpt-4.1-nano",
            "o4-mini",
            "o3",
            "o3-mini",
            "o1",
            "o1-mini",
            "gpt-4o",
            "gpt-4o-mini",
            "gpt-4-turbo",
            "gpt-35-turbo"
          ]
        }
      }
    }
  }
}
Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 9d ago First seen · 70 lines scan A ed045aacb030

Subscribe to this mod's changes

crawl4ai is an MCP server published in the GitHub repository walksoda/crawl-mcp (48 stars, last pushed 2mo ago), licensed MIT. Its token cost is not measured: an MCP server costs its tool schemas, not its config file. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.