Fast Python web crawler for RAG and AI ingestion. Extracts clean Markdown from any site for LLMs and vector stores.
Latest release v0.11.1 — v0.11.1 — default aggregator-page URL filter · 12 May 2026
1 file for Claude Code: markcrawl CLAUDE.md — 86 tokens loaded in every session.
CLAUDE.md A 86 tok These files are AIMLPM/markcrawl's own configuration — they tell Claude Code how to work on this repository, so they are not mods to install elsewhere. Copy one as a starting point and replace the parts that are about this project.