scrapling

scrapling is a skill for Claude Code from fengshao1227/ccg-workflow. It costs 74 tokens per session (1,083 once invoked), scanned A, original, MIT.

A Python-based web-scraping workflow for fetching pages and extracting data, including pages that need JavaScript, login sessions, or protection handling. Web scraping means collecting information from websites by code.

In plain words
What is it for?
It is for downloading and parsing static or dynamic pages, keeping login cookies, handling protected sites, and recording reusable site patterns.
Why use it?
Different websites require different fetching methods, and a simple request may return an error or an empty page.

Skill for Claude Code

Written for Claude Code: allowed-tools in frontmatter.

Part of the ccg plugin — 58 skills, 12 commands, 7 agents shipped together

About the project

CCG is a command-line workflow engine that coordinates Claude, Codex, Gemini, and other models as specialized collaborators on coding tasks. It is used to analyze requests, choose a strategy, delegate work to model-specific roles, and combine their results. The catalogue entries provide the skills, commands, agents, and plugin that implement this workflow.

fengshao1227/ccg-workflow · 5,874 stars · on GitHub · ccg.fengshao1227.com

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/fengshao1227/ccg-workflow/scrapling
Any agent
npx skills add fengshao1227/ccg-workflow --skill scrapling
Clone the repo
git clone --depth 1 https://github.com/fengshao1227/ccg-workflow

Made for: Claude Code.

Or install ccg, the plugin that ships this one along with the rest of its 58 skills, 12 commands, 7 agents.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for scrapling

README.md
[![agentmods](https://agentmods.dev/badge/skills/fengshao1227/ccg-workflow/scrapling.svg)](https://agentmods.dev/skills/fengshao1227/ccg-workflow/scrapling)
Your own site
<a href="https://agentmods.dev/skills/fengshao1227/ccg-workflow/scrapling"><img src="https://agentmods.dev/badge/skills/fengshao1227/ccg-workflow/scrapling.svg" alt="Measured on agentmods" height="20"></a>
Per session 74 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 1,083 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00074 $0.01083
Opus 5 $0.00037 $0.00541
Sonnet 5 $0.00015 $0.00217
Haiku 4.5 $0.00007 $0.00108

Measured 2d ago against content hash e369e2cd5f31, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-06, from the pricing page.

Security

Grade A, and why

scrapling scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

The scan reads SKILL.md. This mod also ships 4 executable files (templates/basic_fetch.py, templates/parse_only.py, templates/session_login.py, …), listed below but not scanned — reading those needs a real analyzer, not pattern matching.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

Origin

Copies of this mod

2 near-identical copies found in the catalogue:

  • scrapling — 100% identical, 0 lines differ
  • scrapling — 100% identical, 0 lines differ
templates/skills/scrapling/SKILL.md · 99 lines

How it starts

The opening of the file, as written. The whole thing — 99 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Scrapling 网页抓取 Skill

步骤 0:检查版本

pip show scrapling
  • 未安装 → 执行 pip install "scrapling[fetchers]" + scrapling install
  • 有新版 → 执行 pip install --upgrade "scrapling[fetchers]" → 查 changelog 告知用户
  • 已最新 → 继续

步骤 1:选择 Fetcher

目标网站 →
│
├─ 已有 HTML 字符串/文件,只需解析?
│   → Selector(纯解析,无网络请求)
│   → 模板: templates/parse_only.py
│
├─ 静态页面,无 JS 渲染,无反爬?
│   → Fetcher(最快,基于 curl_cffi)
│   → 模板: templates/basic_fetch.py
│
├─ 需要登录(HTTP 表单,非 JS 登录)?
│   → FetcherSession(保持会话 cookie)
│   → 模板: templates/session_login.py
│
├─ 有 Cloudflare / WAF 保护?
│   → StealthyFetcher(Camoufox 浏览器,自动过 CF)
│   → 模板: templates/stealth_cloudflare.py
│
├─ SPA 应用(React/Vue),需要 JS 渲染?
│   → DynamicFetcher(Playwright 浏览器)
│   → 基于模板即时生成
│
└─ 不确定?
    → 先用 Fetcher 试,403/空内容 → 升级到 StealthyFetcher

步骤 2:执行工作流

1. 检查版本(步骤 0)
2. 查阅 references/site-patterns.md — 匹配已有模式则直接复用
3. 无匹配 → 用决策树选择 Fetcher
4. 读取对应模板 → 替换参数 → 生成完整脚本
5. 执行脚本 → 返回结果
6. **沉淀经验(必做)**:
   - 新站点 → 追加到 site-patterns.md
   - 新 cookie / 用户提供了 cookie → 保存到 cookie-vault.md
   - **完成抓取后必须检查**:是否有新的 cookie 或 site pattern 需要保存
Fetcher 类型 Cookie 格式 示例
Fetcher / FetcherSession dict {'name': 'value', 'token': 'abc'}
StealthyFetcher / DynamicFetcher list[dict] [{'name': 'n', 'value': 'v', 'domain': '.site.com', 'path': '/'}]

浏览器 Fetcher cookie 必填字段: name, value, domain, path

超时单位速查

Fetcher 类型 超时单位 示例
Fetcher / FetcherSession timeout=30
StealthyFetcher / DynamicFetcher 毫秒 timeout=60000

模板索引

模板 文件 何时读取
基础 HTTP 抓取 templates/basic_fetch.py 目标为静态页面,无反爬
Cloudflare 绕过 templates/stealth_cloudflare.py 目标有 CF/WAF 保护
Session 登录 templates/session_login.py 需 HTTP 表单登录后抓取
纯 HTML 解析 templates/parse_only.py 已有 HTML 字符串,只需提取数据

References 索引

文件 何时读取
references/site-patterns.md 每次抓取前先查阅 — 检查目标站点是否有已记录的模式
references/api-quick-ref.md 生成脚本时查阅 — Fetcher/Selector 方法签名和参数
references/troubleshooting.md 执行报错时查阅 — 按错误信息查找原因和解决方案
references/cookie-vault.md 需要登录 cookie 时查阅 — 检查是否有历史记录可复用
references/maintenance.md 安装/升级/依赖问题时查阅 — 安装层级和验证命令

Read the full file on GitHub · 99 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 99 lines · 74 tokens per session scan A e369e2cd5f31

Subscribe to this mod's changes

scrapling is a skill published in the GitHub repository fengshao1227/ccg-workflow (5,874 stars, last pushed 3d ago), licensed MIT. It adds 74 tokens to every session and 1,083 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.

Related

Other skills, from other repositories