api
01Skill Claude CodeCodex
Use crawlberg api reference documentation.
not rated 165 5d ago A 9 tokens
copy · 88% MIT
165Stars on the repository
19Mods indexed here, across every type
5d agoLast push, which is what freshness is scored on
MITLicence, which decides whether bodies are shown
cli
02Skill Claude CodeCodex
Use crawlberg cli reference documentation.
not rated 165 5d ago A 9 tokens
original MIT
mcp
03Skill Claude CodeCodex
Use crawlberg mcp reference documentation.
not rated 165 5d ago A 11 tokens
original MIT
dev-cycle
04Skill Claude CodeCodex
Crawlberg iteration loops codified as Taskfile tasks — alef install/generate/format/bump, core and binding builds, e2e generate/build/test cycles, cleanup tiers, and the mock-server / stale-.so / precompiled-NIF / generated-e2e gotchas. Load when running or debugging crawlberg build, alef regeneration, or e2e…
not rated 165 5d ago A 89 tokens
original MIT
Skill Claude CodeCodex
Part of crawlberg
Use when extracting a page needs scripted interaction first — click, type, press a key, scroll, wait, screenshot, or run JS before capturing the DOM. Covers crawlberg interact URL --actions with the real action schema, result shape, limits, and external-CDP options.
not rated 165 5d ago A 63 tokens
original MIT
crawlberg
06Skill Claude CodeCodex
Part of crawlberg
Crawl, scrape, and convert websites to Markdown using the local crawlberg CLI and its MCP server. Use when the user wants to fetch a page, follow links across a domain, enumerate URLs, or drive a real browser. Covers installation, the subcommands (scrape, crawl, map, interact, batch-scrape, batch-crawl, download…
not rated 165 5d ago A 107 tokens
original MIT
Skill Claude CodeCodex
Part of crawlberg
Use when the user wants to follow links across a domain and capture every reachable page as Markdown. Covers crawlberg crawl with depth, page caps, concurrency, rate limiting, domain scoping, robots, and output selection.
not rated 165 5d ago A 52 tokens
original MIT
Skill Claude CodeCodex
Part of crawlberg
Use when a static fetch returns nothing useful and the page needs a real browser. Covers --browser-mode auto|always|never, external CDP via --browser-endpoint, symptoms of JS-only pages and WAF blocks, and the performance cost.
not rated 165 5d ago A 57 tokens
original MIT
mapping-urls
09Skill Claude CodeCodex
Part of crawlberg
Use when the user wants the list of URLs on a site rather than the page content — sitemap analysis, link planning, or seeding another tool. Covers crawlberg map URL with --limit, --search, robots, output, and how it differs from a full crawl.
not rated 165 5d ago A 63 tokens
original MIT
Skill Claude CodeCodex
Part of crawlberg
Use when the user wants a single page rendered as clean Markdown plus structured metadata. Covers crawlberg scrape URL, JSON vs Markdown output, what metadata is returned, and how to handle JS-heavy pages.
not rated 165 5d ago A 48 tokens
original MIT
Skill Claude CodeCodex
Part of crawlberg
Use when the user wants a long-running HTTP service for scrape/crawl/map instead of one-shot CLI calls or the MCP server — for example wiring crawlberg into other apps over REST. Covers crawlberg serve, the Firecrawl-v1-compatible endpoints, --host/--port, and when to prefer it.
not rated 165 5d ago A 71 tokens
original MIT
At most 3 mods per repository are shown here, and a mod shipped inside a plugin is left to that plugin's page — the rest are on their repository pages: