Borrowing it
Nothing to install: this file belongs to sadiuysal/crawl4ai-mcp-server. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/sadiuysal/crawl4ai-mcp-server/master/CLAUDE.mdgit clone --depth 1 https://github.com/sadiuysal/crawl4ai-mcp-serverWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/instructions/sadiuysal/crawl4ai-mcp-server/claude-md)<a href="https://agentmods.dev/instructions/sadiuysal/crawl4ai-mcp-server/claude-md"><img src="https://agentmods.dev/badge/instructions/sadiuysal/crawl4ai-mcp-server/claude-md/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/instructions/sadiuysal/crawl4ai-mcp-server/claude-md"><img src="https://agentmods.dev/badge/instructions/sadiuysal/crawl4ai-mcp-server/claude-md.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.04140 | $0.04140 |
| Opus 5 | $0.02070 | $0.02070 |
| Sonnet 5 | $0.00828 | $0.00828 |
| Haiku 4.5 | $0.00414 | $0.00414 |
Grade A, and why
crawl4ai-mcp-server CLAUDE.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 11d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 413 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Project Working Agreements for Claude Code
You are a senior software engineer collaborating on crawler_agent. Your primary goal is to expose Crawl4AI via an MCP server that integrates cleanly with OpenAI Agents SDK and works well in Cursor/Cloud Code.
Mission and scope
- Build a robust stdio MCP server wrapping Crawl4AI (
AsyncWebCrawler) with a minimal tool surface:crawl_url,deep_crawl,adaptive_crawl. - Provide strict JSON schemas for inputs/outputs; add observability and guardrails.
- Wire into OpenAI Agents SDK using
MCPServerStdio; cache tool lists when appropriate.
Workflow
- Confirm scope and propose a short plan before edits.
- Make minimal, reversible changes with small diffs.
- Write/expand tests when feasible; keep coverage meaningful.
- Run minimal validation (build/test/lint/smoke) and capture outputs.
- Summarize changes and propose next steps.
References (authoritative)
- Crawl4AI docs:
https://docs.crawl4ai.com/ - OpenAI Agents SDK (MCP):
https://openai.github.io/openai-agents-python/mcp/ - Tools reference:
https://openai.github.io/openai-agents-python/tools/ - Agent reference (mcp_servers, mcp_config):
https://openai.github.io/openai-agents-python/ref/agent/ - MCP introduction:
https://modelcontextprotocol.io/introduction
Local documentation usage
- Only read targeted excerpts from large files. For
docs/modelcontextprotocol-python-sdk-CLAUDE-MD.txt, search specific symbols/sections; do not ingest the entire file. - Prefer parallel, read-only scans (grep + read specific lines) when gathering context.
Design constraints and NFRs
- Transport: start with stdio MCP; consider SSE/Streamable HTTP later.
- Contracts: strict Pydantic schemas; version tool contracts; deterministic outputs.
- Observability: structured logs; tracing spans for
list_tools/call_tool; basic metrics. - Safety: domain allow/deny lists; robots.txt compliance; depth/TTL/timeouts; proxy/auth constraints; no internal networks.
- Performance: reuse browser contexts; cache where appropriate; avoid unnecessary deep crawls.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 11d ago First seen · 413 lines · 4,140 tokens per session scan A 3e359b92455f
crawl4ai-mcp-server CLAUDE.md is an instructions file published in the GitHub repository sadiuysal/crawl4ai-mcp-server (109 stars, last pushed 7mo ago), licensed MIT. It adds 4,140 tokens to every session, about $0.0207 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other instructions, from other repositories
next.js AGENTS.md
AGENTS.md instructions for vercel/next.js, covering next.js development guide, codebase structure, monorepo overview, core package: packages/next and other important packages.
codex AGENTS.md
AGENTS.md instructions for openai/codex, covering rust/codex-rs, the codex-core crate, code review rules, crate api surface and model visible context.
vscode buildNext.instructions.md
Working notes and architecture documentation for the new esbuild-based build system in build/next. Use when making changes to the new build pipeline (transpile/bundle commands, NLS plugin, source-map handling, resource copying, or self-hosting watch tasks).
spec-kit AGENTS.md
AGENTS.md instructions for github/spec-kit, covering agents.md, about spec kit and specify, quickstart — add a new integration in 5 steps, integration architecture and integrationmanifest — file tracking.
vscode oss-third-party-notices.instructions.md
Instructions for microsoft/vscode, covering vs code oss third-party-notices pipeline, architecture, pipeline flow in ci, applying the notice (cutover) and fallback chain (never fail the build).
langchain AGENTS.md
AGENTS.md instructions for langchain-ai/langchain, covering global development guidelines for the langchain monorepo, corridor security analysis, project architecture and context, monorepo structure and development tools & commands.