unclecode/crawl4ai

🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN

About the project

Crawl4AI is an open-source web crawler and scraper that converts online content into clean Markdown for language-model retrieval, agents, and data pipelines. It is used for web extraction at scale where structured, LLM-ready content is needed. The catalogue includes a command for working with Crawl4AI.

Latest release v0.9.3 — Release v0.9.3 · 31 Aug 2026

These files are unclecode/crawl4ai's own configuration. They tell Claude Code how to work on this repository, so they are not mods to install elsewhere. Copy one as a starting point and replace the parts that are about this project.

81,675Stars on the repository
1Files it configures its agents with
Tokens loaded in every session
1Agent configured

Commands