crawler skills

180 tagged crawler, measured the same way as everything else here.

Browse within: Multi-Agent 105digital-employee 105makemoney 105mas 105multi-agent-systems 105browser-automation 42bun 42cdp 42curl-impersonate 42audit 35cli 35core-web-vitals 35data-extraction 34scraper 21

firecrawl-build

01

firecrawl/firecrawl

Skill Claude CodeCodex

Integrate Firecrawl into application code whenever a product, agent, or workflow needs web data inside the app — web search, live search results, page scraping, structured extraction, or browser interaction. Use when building any feature that needs data from the web in code, even if the user does not mention Firecrawl…

not rated 192k +2.0k today A SkillSpector: warn 149 tokens AGPL-3.0

scrapling-official

02

D4Vinci/Scrapling

Skill Claude CodeCodex

Scrape web pages using Scrapling with anti-bot bypass (like Cloudflare Turnstile), stealth headless browsing, spiders framework, adaptive scraping, and JavaScript rendering. Use when asked to scrape, crawl, or extract data from websites; webfetch fails; the site has anti-bot protections; write Python code to…

not rated 88k +1.2k today A ✓ AI review SkillSpector: warn 80 tokens original BSD-3-Clause

TeamWiseFlow/xiaobei

Skill Claude CodeCodex

A browser-based workflow for publishing videos to WeChat Channels, a Chinese short-video platform, using a saved browser session.

not rated 8.6k +17 changed 6d ago A SkillSpector: pass 44 tokens

node-crawler

04

bda-research/node-crawler

Skill Claude CodeCodex

Node.js web crawler for production-grade, large-scale tasks — NOT for simple one-off requests. Use only when: bulk scraping, batch downloading, multi-page crawling, long-running spiders, or complex multi-step workflows that need connection pooling, rate limiting, proxy rotation, automatic retries, and Cheerio parsing.…

not rated 6.8k +2 3mo ago A SkillSpector: pass 115 tokens original MIT

boss-zhipin-scraper

05

eatmoreduck/boss-zhipin-scraper

Skill Claude CodeCodex

A tool for collecting job listings from BOSS直聘, a Chinese job-search website, through a logged-in Chrome browser. It outputs structured JSON or CSV data and can summarize the results.

not rated 1.6k +39 changed yesterday A 78 tokens original MIT

webreaper

06

alex-on-ai/WebReaper

Skill Claude CodeCodex

Scrape, crawl, or extract structured data from one or more URLs via the webreaper CLI. Outputs clean Markdown by default; JSON when a schema is given, or schema-free JSON when an LLM endpoint is configured (extract by natural-language prompt, no CSS selectors). Maps a site's URLs in one call. Handles JS-rendered pages…

not rated 150 +1 changed 11d ago C 413 tokens original MIT

NatsuFox/ClaudeChrome

Skill Codex

Extract API endpoints from the active ClaudeChrome browser tab and save them as local artifacts. Use when the user asks to find, capture, reverse engineer, document, or generate Python scripts for endpoints used by the current tab, this page, a live website, or a user-specified browser interaction in ClaudeChrome.

not rated 80 1mo ago A 70 tokens

crawl4weibo

08

Praeviso/crawl4weibo

Skill Claude CodeCodex

Query Weibo users, posts, and comments through the local crawl4weibo CLI. Use when the workspace contains this repository and the user wants deterministic shell access to the project's Weibo crawling capabilities from OpenClaw.

not rated 43 +1 12d ago A SkillSpector: pass 49 tokens original MIT

jsrip-scan

09

mouteee/jsrip

Skill Claude CodeCodex

Use when user asks to scan a target for exposed JavaScript secrets, run a jsrip scan, analyze findings for true/false positives, or says /jsrip-scan. Single-target bug bounty recon — runs jsrip then auto-classifies every finding.

not rated 29 +1 1mo ago A 57 tokens CC-BY-SA-4.0

deepspider

10

ma-pony/deepspider

Skill Claude CodeCodex

Use when tracing browser requests, recovering JavaScript-generated request outputs, diagnosing anti-bot runtime gaps, or delivering independently verified scraping modules.

not rated 21 +1 changed 27d ago A 32 tokens

crawl4ai-skill

11

lancelin111/crawl4ai-skill

Skill Claude CodeCodex

A web crawler and scraper that searches with DuckDuckGo, fetches individual pages or whole sites, and returns clean Markdown intended for language models. It can also scrape pages that need JavaScript to display their content.

not rated 16 7mo ago A 63 tokens original MIT

rust-native-scanner

12

aphrody-code/bxc

Skill Claude CodeCodex

Expertise in high-performance native codebase scanning and validation using Rust. Trigger this skill when asked to "scan the codebase natively", "run rust scanner", or "validate bxc core using rust".

not rated 3 1mo ago A 47 tokens original Apache-2.0

docs

13

mysleekdesigns/crawlforge-mcp

Skill Claude CodeCodex

Scale up scraping and drive interactive pages. Use batchscrape for many URLs, getbatchresults to page through async output, scrapewithactions to interact before scraping, and generatellmstxt to produce a site's AI policy file.

not rated 2 changed 4d ago A 0 tokens original MIT

site-mirror

14

ajmaluk/site-mirror-skill

Skill Claude CodeCodex

Mirror an entire website into a local directory using open‑source scripts. Supports static mirroring via wget, dynamic mirroring for single pages, and dynamic multi‑page crawling via Playwright. These modes capture JavaScript‑loaded assets, rewrite links for offline browsing and can optionally follow internal links to…

not rated 2 4mo ago A 69 tokens original MIT

codex-seo

15

CaffNoir59/codex-seo

Skill Claude CodeCodex

Native Codex SEO audit orchestrator for complete website or page analysis using shared audit context, specialized Agent Skills, Node.js ESM scripts, deterministic JSON, and local HTML/PDF reports. Use for SEO audits, technical SEO, content quality, schema, sitemap, image SEO, GEO or AI-search readiness, Core Web…

not rated 0 2mo ago A 83 tokens

At most 3 mods per repository are shown here, and a mod shipped inside a plugin is left to that plugin's page — the rest are on their repository pages: