28 MCP tools that give Claude, Cursor and any MCP client the live web — scrape, crawl, search, real Google rank, change tracking, document parsing, plus an autonomous agent that researches from a plain-English prompt with no URLs. Clean Markdown and schema-validated JSON, not raw HTML. Local-Ollama extraction by default. MIT, 1,000 free credits.
Latest release v6.3.1 — v6.3.1 — crawlforge login works on Linux · 7 Sept 2026
19 files for Claude Code: api-documenter, deployment-manager, mcp-implementation, performance-monitor and 15 more — 4,192 tokens loaded in every session.
CLAUDE.md C 4,192 tok .claude/skills/agent-creator/SKILL.md A 39 tok .claude/skills/api-documenter/SKILL.md A 40 tok .claude/skills/deployment-manager/SKILL.md A 39 tok .claude/skills/mcp-implementation/SKILL.md A 44 tok .claude/skills/next-phase/SKILL.md A 105 tok .claude/skills/performance-monitor/SKILL.md A 41 tok .claude/skills/project-manager/SKILL.md A 49 tok .claude/skills/security-auditor/SKILL.md A 45 tok .claude/skills/teammate-tool/SKILL.md A 48 tok .claude/skills/teammate-workflow/SKILL.md A 53 tok .claude/skills/testing-validation/SKILL.md A 36 tok .claude/agents/api-documenter.md A 49 tok .claude/agents/deployment-manager.md A 52 tok .claude/agents/mcp-implementation.md A 50 tok .claude/agents/performance-monitor.md A 57 tok .claude/agents/project-manager.md A 62 tok .claude/agents/security-auditor.md A 50 tok .claude/agents/testing-validation.md A 44 tok These files are mysleekdesigns/crawlforge-mcp's own configuration — they tell Claude Code how to work on this repository, so they are not mods to install elsewhere. Copy one as a starting point and replace the parts that are about this project.