A webpage-fetching skill that tries several methods in sequence, from direct requests to browser rendering, to retrieve online content.

Use it to fetch static pages, JavaScript-rendered sites, and login-required content, with different methods selected for known domains.

It handles pages that are blocked, empty, built with JavaScript, or require a logged-in browser session.

smart-fetch — Skill for Claude Code and Codex by Mousaee/agent-search

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent

npx skills add Mousaee/agent-search --skill smart-fetch

Clone the repo

git clone --depth 1 https://github.com/Mousaee/agent-search

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/skills/mousaee/agent-search/smart-fetch/github.svg)](https://agentmods.dev/skills/mousaee/agent-search/smart-fetch)

Your own site

<a href="https://agentmods.dev/skills/mousaee/agent-search/smart-fetch"><img src="https://agentmods.dev/badge/skills/mousaee/agent-search/smart-fetch/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/skills/mousaee/agent-search/smart-fetch"><img src="https://agentmods.dev/badge/skills/mousaee/agent-search/smart-fetch.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 65 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.

When invoked 615 The whole file, excluding the scripts and references it only reads on demand.

Security scan A 1 finding. A grade says what 26 rules found in the file — not that it is safe.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.00065	$0.00615
Opus 5.5	$0.00026	$0.00246
Sonnet 5.5	$0.00013	$0.00123
Haiku 4.5	$0.00006	$0.00061

Measured 2mo ago against content hash 476e90e8a3f9, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

smart-fetch scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

# 或 curl

skills/smart-fetch/SKILL.md · 73 lines

What it actually says

Smart Fetch — 四层降级网页抓取

当需要抓取网页内容时，按以下顺序尝试，每层失败自动降级：

Layer 1: web_fetch（最快）

直接 HTTP 请求，适合静态页面和大部分网站。

# 使用 Agent 内置的 web_fetch 工具
# 或 curl
curl -s "URL" -H "User-Agent: Mozilla/5.0"

成功标志：返回内容 > 200 字符
失败条件：error / 内容为空 / 403/404

Layer 2: Jina Reader（JS 渲染）

适合需要 JavaScript 渲染的页面。

curl -s "https://r.jina.ai/URL" -H "Accept: text/markdown"

成功标志：返回 markdown 内容 > 200 字符
失败条件：内容为空 / 超时

Layer 3: Browser Snapshot（需要登录态/交互）

使用浏览器工具的 snapshot 模式。

browser(action=navigate, targetUrl=URL, profile=openclaw)
browser(action=snapshot, compact=true, profile=openclaw)

成功标志：snapshot 包含有意义的文本
失败条件：页面空白 / 需要验证码

Layer 4: Browser Full Render（最后手段）

完整浏览器渲染，包括等待、滚动、点击展开。仅在前三层全部失败时使用。

决策逻辑

URL 输入
  ├─ 已知需要登录的域名 → 直接 Layer 3
  ├─ 已知 Jina Reader 效果好的域名 → 直接 Layer 2
  └─ 其他 → Layer 1 开始，逐层降级

域名路由表

域名模式	推荐层级	原因
.feishu.cn/wiki/	Layer 3	需要登录态
.feishu.cn/docx/	Layer 3	需要登录态
arxiv.org	Layer 1	静态页面
github.com	Layer 1	静态页面
x.com / twitter.com	Layer 2	JS 渲染
mp.weixin.qq.com	Layer 2	JS 渲染

随着使用不断补充此表。Agent 可以自行维护路由表，记录哪些域名在哪一层成功。

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

2mo ago First seen · 73 lines · 65 tokens per session scan A 476e90e8a3f9

Subscribe to this mod's changes

smart-fetch is a skill published in the GitHub repository Mousaee/agent-search (25 stars, last pushed 8mo ago), licensed MIT. It adds 65 tokens to every session and 615 once invoked, about $0.0003 per session on Opus 5.5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Browser automation Research

smart-fetch next to 6 skills from other repositories
Mod	Type	Stars	Last push	Licence	Scan	Tokens	Works with
smart-fetchMousaee/agent-search	Skill	25	8mo ago	MIT	A	65	Claude Code, Codex
interactive-loginyc-software/qm	Skill	16k	yesterday	MIT	A	46	Claude Code, Codex
pinchtab-mcppinchtab/pinchtab	Skill	11k	10d ago	MIT	A	52	Claude Code, Codex
azure-messaging-webpubsub-javamicrosoft/skills	Skill	3.2k	yesterday	MIT	A	43	Claude Code
cua-driverdavidondrej/skills	Skill	4.3k	3d ago	MIT	A	68	Claude Code, Codex
open-design-browsercompozy/compozy	Skill	2.8k	yesterday	MIT	A	56	Claude Code, Codex
southwestborski/travel-hacking-toolkit	Skill	895	yesterday	MIT	A	40	Claude Code

smart-fetch