web-content-extraction

A tool-assisted procedure for extracting the main text from web pages and formatting it as Markdown or JSON. It is intended for documentation, articles, blogs, news, and open-source project pages.

In plain words
What is it for?
Use it to fetch a public URL or parse a local HTML file, then inspect the extracted title, author, dates, warnings, and Markdown content.
Why use it?
It removes navigation and other page clutter, giving an AI cleaner content to read and summarize.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/cloudnative-co/claude-code-starter-kit/web-content-extraction
Any agent
npx skills add cloudnative-co/claude-code-starter-kit --skill web-content-extraction
Clone the repo
git clone --depth 1 https://github.com/cloudnative-co/claude-code-starter-kit

Made for: Claude Code, Codex.

Per session 110 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 2,280 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00110 $0.02280
Opus 5 $0.00055 $0.01140
Sonnet 5 $0.00022 $0.00456
Haiku 4.5 $0.00011 $0.00228

Measured 2d ago against content hash 7fe25cd5dad2, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

web-content-extraction scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

The scan reads SKILL.md. This mod also ships 12 executable files (scripts/defuddle-file.mjs, scripts/defuddle-url.mjs, scripts/lib/defuddle-core.mjs, …), listed below but not scanned — reading those needs a real analyzer, not pattern matching.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/web-content-extraction/SKILL.md · 127 lines

How it starts

The opening of the file, as written. The whole thing — 127 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Web Content Extraction

Purpose

Webページ本文を抽出し、LLMが読みやすい Markdown/JSON に整形する。 Claude Code が Webページ、URL、公式ドキュメント、ブログ記事、ニュース記事、OSSページを読む場合は、原則として毎回このSkillを使う

Mandatory Rule

When reading any public web URL, use Defuddle first.

Do not summarize, analyze, compare, or review a web page from raw HTML unless Defuddle extraction fails or the page type is explicitly unsupported.

Use Cases

  • URL要約 / 公式ドキュメント確認 / OSS調査
  • ブログ記事分析 / ニュース記事分析 / 競合サイト分析
  • ベンダー公式ブログの調査 / Web上の一次情報確認
  • 技術記事の読み取り / LLM・RAG向けのWeb本文抽出

Standard Commands

# 公開URLを取得して本文をMarkdown/JSON化(SSRFガードあり)
~/.claude/skills/web-content-extraction/scripts/run-node.sh \
  ~/.claude/skills/web-content-extraction/scripts/defuddle-url.mjs <url>
# ローカルHTMLファイルを本文抽出(外部通信なし)
~/.claude/skills/web-content-extraction/scripts/run-node.sh \
  ~/.claude/skills/web-content-extraction/scripts/defuddle-file.mjs <file>

出力は JSON(stdout)。最低限 success, url, fetchedAt/parsedAt, title, author, site, domain, published, description, wordCount, content(Markdown) を含む。 warnings / fetchWarnings がある場合は抽出の信頼性に注意する。

Output Fields

フィールド 意味
success 本文抽出に成功したか(false は抽出失敗/空)
warnings 本文が短い・空など低信頼の警告
url / requestedUrl / finalUrl 対象URL(リダイレクト後の最終URL含む)
fetchedAt / parsedAt 取得・解析時刻(ISO8601, 監査用に必ず保持)
title author site domain published description メタデータ
wordCount 語数(空白区切り。日本語は極端に小さく出る
charCount 非空白の文字数(日本語の実分量はこちらで判断
cjkCharCount CJK文字数(日本語/中国語/韓国語の量の目安)
content 本文(HTMLはMarkdown、PDFはプレーンテキスト)
extractorType サイト固有抽出器が使われた場合の種別
extractorEngine PDF抽出時のみ "pdf"pageCount も付く

Security Rules

  • 同期コア + 非フェッチDOM で動作する(useAsync はupstreamに存在しないため意図を構造で担保)。
  • 外部フォールバック・サブリソース外部取得・ページ内スクリプト実行は行わない
  • 社内URL、顧客URL、認証付きURL、個人情報・機密を含むページを外部送信しない。
  • localhost / プライベートIP(10/8,172.16/12,192.168/16,127/8,169.254/16,100.64/10 等) / .local/.internal 等 / 単一ラベルの内部ホスト名 / 非http(s) / 認証情報付きURL標準で拒否
  • IP判定はバイト単位。10進/8進/16進 IPv4 も拒否。IPv6はdefault-deny(グローバルユニキャスト 2000::/3 以外は全拒否。Teredo/site-local/documentation/NAT64/IPv4-mapped/6to4(private埋め込み)等を含む)。
  • 接続IPをpin(guarded undici dispatcher)して DNSリバインディング/TOCTOU を封じる
  • リダイレクトは手動追従し各ホップを送信前に再検査本文はストリームでサイズ上限(メモリDoS対策)。
  • 開発用途で明示的に許可する場合のみ ALLOW_PRIVATE_URLS=true(バイパスは stderr に監査記録)。
  • 抽出結果には必ず URL と取得日時を残す。
  • 抽出結果だけを唯一の真実として扱わない。重要な事実は一次情報で再確認する。

Read the full file on GitHub · 127 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 127 lines · 0 tokens per session scan A 7fe25cd5dad2

Subscribe to this mod's changes

web-content-extraction is a skill published in the GitHub repository cloudnative-co/claude-code-starter-kit (147 stars, last pushed 9d ago), licensed MIT. It adds 110 tokens to every session and 2,280 once invoked, about $0.0006 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

systematic-debugging

Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes.

obra/superpowers · 21 tokens

chat-pet-sprite-creation

Use when creating or changing VS Code chat pet sprite art, sprite sheets, state animations, eye treatments, Stable/Insiders variants, or pet transitions under src/vs/workbench/contrib/chat/browser/widget/media/chatPet.

microsoft/vscode · 53 tokens

cpu-profile-analysis

Analyze V8/Chrome CPU profiles (.cpuprofile) and DevTools trace files (Trace-.json). Use when: profiling performance, investigating slow functions, comparing code paths, finding bottlenecks, analyzing timeToRequest, understanding call trees from sampling profiler data, analyzing layout/paint/rendering, investigating…

microsoft/vscode · 71 tokens

babysit-pr

Babysit a GitHub pull request after creation by continuously polling review comments, CI checks/workflow runs, and mergeability state until the PR is merged/closed or user help is required. Diagnose failures, retry likely flaky failures up to 3 times, auto-fix/push branch-related issues when appropriate, and keep…

openai/codex · 114 tokens

imagegen

Generate or edit raster images when the task benefits from AI-created bitmap visuals such as photos, illustrations, textures, sprites, mockups, or transparent-background cutouts. Use when Codex should create a brand-new image, transform an existing image, or derive visual variants from references, and the output…

openai/codex · 113 tokens

agent-host-chat-contributions

Build and review cross-cutting agent-host chat behavior through lifecycle contributions. Use when adding turn lifecycle side effects, prompt or context injection, restored-history transformation, protocol-action observation, or when reviewing changes that add code to AgentSideEffects or AgentService.

microsoft/vscode · 56 tokens