web-content-fetcher

web-content-fetcher is a skill for Claude Code, Codex from UnicomAI/wanwu. It costs 83 tokens per session (516 once invoked), scanned A, original, Apache-2.0.

A webpage content fetcher that tries alternative services to turn web pages into Markdown, including pages blocked by some crawler filters.

In plain words
What is it for?
Use it to fetch a webpage, convert it to Markdown, or try alternative fetching methods for filtered or protected sites.
Why use it?
It provides fallback ways to retrieve readable page content when a normal web fetch does not work.

Skill for Claude CodeCodex

Which agent this was written for is unclear — built for openclaw. Also seen: built for openclaw.

Good fit Use it to fetch a webpage, convert it to Markdown, or try alternative fetching methods for filtered or protected sites.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/unicomai/wanwu/web-content-fetcher
About the project

Wanwu is an enterprise platform for building AI agents, workflows, retrieval-augmented applications, and managing models in multi-tenant environments. It is designed for developers and enterprise teams delivering AI applications and integrations. The catalogue entries provide skills and agents for using the platform.

UnicomAI/wanwu · 2,461 stars · on GitHub

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add UnicomAI/wanwu --skill web-content-fetcher
Clone the repo
git clone --depth 1 https://github.com/UnicomAI/wanwu

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for web-content-fetcher

README.md
[![agentmods](https://agentmods.dev/badge/skills/unicomai/wanwu/web-content-fetcher/github.svg)](https://agentmods.dev/skills/unicomai/wanwu/web-content-fetcher)
Your own site
<a href="https://agentmods.dev/skills/unicomai/wanwu/web-content-fetcher"><img src="https://agentmods.dev/badge/skills/unicomai/wanwu/web-content-fetcher/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for web-content-fetcher

Your own site · 80×15
<a href="https://agentmods.dev/skills/unicomai/wanwu/web-content-fetcher"><img src="https://agentmods.dev/badge/skills/unicomai/wanwu/web-content-fetcher.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 83 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 516 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 1 finding. A grade says what 26 rules found in the file — not that it is safe. Third-party audits
  • NVIDIA SkillSpector pass 7 Sept 2026
How audits are shown
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00083 $0.00516
Opus 5 $0.00042 $0.00258
Sonnet 5 $0.00017 $0.00103
Haiku 4.5 $0.00008 $0.00052

Measured 6d ago against content hash 9faa4be2a694, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-09, from the pricing page.

Security

Grade A, and why

web-content-fetcher scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 6d ago.

The scan reads SKILL.md. This mod also ships 2 executable files (fetch_content.py, fetch.sh), listed below but not scanned — reading those needs a real analyzer, not pattern matching.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

curl -s "https://r.jina.ai/https://example.com"
configs/microservice/bff-service/configs/agent-skills/clawhub/web-content-fetcher/SKILL.md · 72 lines

What it actually says

网页内容获取工具

当常规 web_fetch/web_search 无法获取内容时,使用替代服务获取网页 Markdown 格式内容。

支持的服务

优先级 服务 用法 适用场景
1 r.jina.ai https://r.jina.ai/{url} 最稳定,通用性强
2 markdown.new https://markdown.new/{url} Cloudflare 保护网站
3 defuddle.md https://defuddle.md/{url} 备用方案

使用方法

直接调用

当需要获取网页内容时,按顺序尝试:

  1. 首先用 web_fetch 尝试获取
  2. 如果失败或被过滤,调用本工具
# 使用 jina.ai (首选)
curl -s "https://r.jina.ai/https://example.com"

# 使用 markdown.new (Cloudflare)
curl -s "https://markdown.new/https://example.com"

# 使用 defuddle.md (备用)
curl -s "https://defuddle.md/https://example.com"

API 格式

# 简单获取
fetch_webpage <url>

# 指定方法
fetch_webpage <url> --method jina|markdown|defuddle

示例

用户: 帮我获取 https://news.example.com/article/123 的内容
助手: (使用 r.jina.ai 获取)

工具脚本

本目录包含 fetch.sh 脚本,可直接调用:

./fetch.sh https://example.com
./fetch.sh https://example.com jina

让网页内容获取不再受限 🌐

Files

What ships with it

4 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 6d ago First seen · 72 lines · 83 tokens per session scan A 9faa4be2a694

Subscribe to this mod's changes

web-content-fetcher is a skill published in the GitHub repository UnicomAI/wanwu (2,461 stars, last pushed 5d ago), licensed Apache-2.0. It adds 83 tokens to every session and 516 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.

Related

Other skills, from other repositories

dify

Use when building LLM applications with visual workflow — RAG knowledge bases, AI agents, chatbots with drag-and-drop orchestration. Dify: open-source LLM app platform supporting 30+ models (OpenAI, Claude, DeepSeek, Ollama, Qwen, GLM) with Docker deployment.

znlgis/opengis-skills · 66 tokens

ai-skills

Use when building LLM applications, RAG knowledge bases, AI agents, terminal coding agents, multi-model orchestration, plugin-based agent harnesses, or file translation. Index of 10 skills: Dify, Hermes Agent, OpenClaw, OpenCode, Pi, DocuTranslate, Oh-My-OpenAgent, Superpowers-zh, DeepSeek Harness, My OpenCode…

znlgis/opengis-skills · 86 tokens

Web2Skill

Convert one public website URL or an explicit batch of public URLs into a reusable skill zip backed by rendered HTML snapshots and a bounded JSONL retrieval index. Use to discover a documentation directory from one URL, crawl a supplied URL set sequentially, generate a source profile, or package indexed web content as…

zhimaAi/chatwiki · 67 tokens

Book2Skill

Convert one or more TXT, Markdown, DOCX, or PDF documents into a reusable skill zip backed by normalized Markdown, extracted images, and a grounded JSONL knowledge index. Invoke this skill before inspecting task files, then execute its workflow directly without listing directories.

zhimaAi/chatwiki · 56 tokens

agent-orchestration-multi-agent-optimize

Optimize multi-agent systems with coordinated profiling, workload distribution, and cost-aware orchestration. Use when improving agent performance, throughput, or reliability.

ranbot-ai/awesome-skills · 38 tokens

ai-agents-architect

Expert in designing and building autonomous AI agents. Masters tool use, memory systems, planning strategies, and multi-agent orchestration.

ranbot-ai/awesome-skills · 31 tokens