web-extract

web-extract is a skill for Claude Code, Codex from ZaptainZ/radioheader. It costs 97 tokens per session (1,273 once invoked), scanned A, original, MIT.

A skill for extracting the main text from a web page and returning it as clean Markdown. It uses several fallback methods when the first method cannot access or parse the page.

In plain words
What is it for?
Fetch and read articles or other page content from a supplied URL, including selected sites such as GitHub, documentation pages, and WeChat.
Why use it?
It removes navigation, ads, and other page clutter from web content, while providing alternatives for sites that block or complicate direct access.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/zaptainz/radioheader/web-extract
Any agent
npx skills add ZaptainZ/radioheader --skill web-extract
Clone the repo
git clone --depth 1 https://github.com/ZaptainZ/radioheader

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for web-extract

README.md
[![agentmods](https://agentmods.dev/badge/skills/zaptainz/radioheader/web-extract.svg)](https://agentmods.dev/skills/zaptainz/radioheader/web-extract)
Your own site
<a href="https://agentmods.dev/skills/zaptainz/radioheader/web-extract"><img src="https://agentmods.dev/badge/skills/zaptainz/radioheader/web-extract.svg" alt="Measured on agentmods" height="20"></a>
Per session 97 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 1,273 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00097 $0.01273
Opus 5 $0.00048 $0.00636
Sonnet 5 $0.00019 $0.00255
Haiku 4.5 $0.00010 $0.00127

Measured 4d ago against content hash 6049ca935f3e, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

web-extract scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.

The scan reads SKILL.md. This mod also ships 1 executable file (scrapling_fetch.py), listed below but not scanned — reading those needs a real analyzer, not pattern matching.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

templates/skills/web-extract/SKILL.md · 126 lines

How it starts

The opening of the file, as written. The whole thing — 126 lines — stays where its author put it; the contents beside it link to each section on GitHub.

概述

从任意 URL 提取干净的 Markdown 正文,自动选择最优方案。统一 maxChars=30000。

输入

用户提供的 URL 在 $ARGUMENTS 中。如果没有提供 URL,询问用户。

执行策略

域名快捷路由(优先判断)

域名模式 直接走 原因
mp.weixin.qq.com Scrapling Jina 403,不浪费配额
github.com web_fetch 静态页,无需 Jina
raw.githubusercontent.com web_fetch 原始文件
docs.* / *.readthedocs.io web_fetch 技术文档,静态

非快捷路由 → 三层降级

第一层:Jina Reader(优先)

使用 WebFetch 工具访问: https://r.jina.ai/{原始URL}
设置 maxChars=30000

Jina 自动渲染、抽取正文、去噪,返回干净 Markdown。速度约 1.4 秒。

  • 每天免费 200 次
  • 如果返回内容为空、403、或明显不完整 → 降级到第二层

第二层:Scrapling + html2text(降级)

~/.claude/skills/web-extract/.venv/bin/python3 ~/.claude/skills/web-extract/scrapling_fetch.py "<URL>" 30000

依赖已安装在 skill 自带的 venv 中,无需额外操作。

特点:

  • 无限制,无需 API Key
  • 配合 html2text 输出与 Jina 同等质量的 Markdown
  • 如果脚本报错(依赖未装等)→ 降级到第三层

第三层:web_fetch 直接抓(兜底)

使用 WebFetch 工具直接访问原始 URL
设置 maxChars=30000

注意:

  • 返回全页 HTML 转 Markdown,噪音多
  • 有反爬的平台会直接失败
  • 仅适合静态页面

验证拦截检测 + Cookie 辅助(第四层)

当以上三层返回的内容包含以下特征时,判定为被验证拦截:

  • 内容含"环境异常"、"完成验证"、"appmsgcaptcha"等字样
  • 返回内容为空或明显是验证页(而非正文)
  • Scrapling 报超时且域名为 mp.weixin.qq.com 等已知需验证站点

检测到拦截后,必须告知用户并请求确认:

该网站需要人机验证,常规提取方式无法获取正文。 我可以尝试使用你本地浏览器(Chrome/Safari)中已有的 cookie 来绕过验证。 这会读取你浏览器中该站点的 cookie(仅用于本次请求,不会存储或外传)。

是否允许使用浏览器 cookie 提取?

用户确认后,使用 --cookie 标志调用:

~/.claude/skills/web-extract/.venv/bin/python3 ~/.claude/skills/web-extract/scrapling_fetch.py "<URL>" 30000 --cookie

用户拒绝时,告知替代方案:

  • 在浏览器中手动打开文章,复制正文粘贴到对话中
  • 或将文章链接转发给自己后用其他阅读器打开

已知需要 cookie 的站点

域名 原因
mp.weixin.qq.com 微信公众号文章,2026 年起要求 PoC 验证码

遇到新的需验证站点时,同样走「检测 → 提示 → 确认 → cookie」流程。

输出格式

提取成功后,向用户展示:

  1. 使用的方案:告知用了哪一层(Jina / Scrapling / web_fetch / Cookie)
  2. 正文内容:干净的 Markdown
  3. 字符数:实际提取的字符数
  4. 如果发生降级或使用了 cookie,简要说明原因

依赖

Scrapling、html2text、browser-cookie3 已安装在 skill 自带的 venv 中:~/.claude/skills/web-extract/.venv/

Read the full file on GitHub · 126 lines

Files

What ships with it

1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 4d ago First seen · 126 lines · 97 tokens per session scan A 6049ca935f3e

Subscribe to this mod's changes

web-extract is a skill published in the GitHub repository ZaptainZ/radioheader (22 stars, last pushed 1mo ago), licensed MIT. It adds 97 tokens to every session and 1,273 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

systematic-debugging

Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes.

obra/superpowers · 21 tokens

brainstorming

You MUST use this before any creative work - creating features, building components, adding functionality, or modifying behavior. Explores user intent, requirements and design before implementation.

obra/superpowers · 37 tokens

auto-perf-optimize

Run agent-driven VS Code performance or memory investigations. Use when asked to launch Code OSS, automate a VS Code scenario, run the Chat memory smoke runner, capture renderer heap snapshots, take workflow screenshots, compare run summaries, or drive a repeatable scenario before heap-snapshot analysis.

microsoft/vscode · 62 tokens

chat-perf

Run chat perf benchmarks and memory leak checks against the local dev build or any published VS Code version. Use when investigating chat rendering regressions, validating perf-sensitive changes to chat UI, or checking for memory leaks in the chat response pipeline.

microsoft/vscode · 51 tokens

chat-pet-sprite-creation

Use when creating or changing VS Code chat pet sprite art, sprite sheets, state animations, eye treatments, Stable/Insiders variants, or pet transitions under src/vs/workbench/contrib/chat/browser/widget/media/chatPet.

microsoft/vscode · 53 tokens

cpu-profile-analysis

Analyze V8/Chrome CPU profiles (.cpuprofile) and DevTools trace files (Trace-.json). Use when: profiling performance, investigating slow functions, comparing code paths, finding bottlenecks, analyzing timeToRequest, understanding call trees from sampling profiler data, analyzing layout/paint/rendering, investigating…

microsoft/vscode · 71 tokens