What is fetch-url-as-markdown?

A web-page reader that downloads a URL and returns the main content as clean Markdown, a plain-text format commonly used in code and documentation.

Use it to fetch, read, scrape, summarize, or quote web pages. It is not intended for PDFs, images, or archive files.

It strips away navigation, footers, and advertisements, and can use a fallback when a page needs JavaScript or blocks normal downloads.

fetch-url-as-markdown — Skill for Claude Code by CodeAlive-AI

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent

npx skills add CodeAlive-AI/ai-driven-development --skill fetch-url-as-markdown

Clone the repo

git clone --depth 1 https://github.com/CodeAlive-AI/ai-driven-development

Made for: Claude Code.

Or install ai-driven-development, the plugin that ships this one along with the rest of its 22 skills, 1 agent.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/skills/codealive-ai/ai-driven-development/fetch-url-as-markdown/github.svg)](https://agentmods.dev/skills/codealive-ai/ai-driven-development/fetch-url-as-markdown)

Your own site

<a href="https://agentmods.dev/skills/codealive-ai/ai-driven-development/fetch-url-as-markdown"><img src="https://agentmods.dev/badge/skills/codealive-ai/ai-driven-development/fetch-url-as-markdown/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/skills/codealive-ai/ai-driven-development/fetch-url-as-markdown"><img src="https://agentmods.dev/badge/skills/codealive-ai/ai-driven-development/fetch-url-as-markdown.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 90 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.

When invoked 953 The whole file, excluding the scripts and references it only reads on demand.

Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe. Third-party audits

NVIDIA SkillSpector pass 7 Sept 2026

How audits are shown

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.00090	$0.00953
Opus 5.5	$0.00036	$0.00381
Sonnet 5.5	$0.00018	$0.00191
Haiku 4.5	$0.00009	$0.00095

Measured 1mo ago against content hash 075bf66f381b, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

fetch-url-as-markdown scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.

The scan reads SKILL.md. This mod also ships 1 executable file (scripts/fetch_url.py), listed below but not scanned — reading those needs a real analyzer, not pattern matching.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/fetch-url-as-markdown/SKILL.md · 83 lines

How it starts

The opening of the file, as written. The whole thing — 83 lines — stays where its author put it; the contents beside it link to each section on GitHub.

URL to Markdown

Fetch any web URL and get clean, readable Markdown — main content only, no navigation/footer/ads. Local + free by default; smart fallback to Exa MCP when the page can't be extracted locally.

Workflow (the only thing the agent needs to remember)

Try trafilatura first:

python3 ~/.claude/skills/fetch-url-as-markdown/scripts/fetch_url.py "<URL>"

If exit code is 1 or 2 → fall back to Exa MCP with the same URL:
```
mcp__exa__web_search_advanced_exa(
    query="<URL>",
    includeDomains=["<host of URL>"],
    numResults=1,
    textMaxCharacters=50000,
    type="auto"
)
```
(mcp__exa__crawling works too if the server exposes it; the web_search_advanced_exa call above is the always-available variant — pin the host with includeDomains and use the URL itself as the query.)
Exit code 3 means trafilatura is not installed — install once:
```
python3 -m pip install --break-system-packages trafilatura
```

Exit codes (what they mean for the fallback decision)

Code	Meaning	Action
0	Markdown printed to stdout	done
1	DownloadError — network/HTTP/timeout/anti-bot block at fetch	fall back to Exa
2	ExtractionError — empty extract, JS/Cloudflare wall, or stub body (<200 chars)	fall back to Exa
3	trafilatura missing	install (see above), then retry
4	UnsupportedContentTypeError — URL is binary (PDF, image, archive)	don't fall back to Exa; use the right specialized skill (e.g. `pdf` for PDFs)

Defaults baked into the script

output_format="markdown", include_formatting=True — keeps headings/lists/code structure where the source HTML uses real <h1..h6> etc.
include_links=True, include_tables=True
with_metadata=True → emits a YAML frontmatter (title, author, date, url, hostname)
favor_recall=True, deduplicate=True — readable but trims duplicates
Real-browser User-Agent + 30s timeout configured in scripts/settings.cfg
Anti-stub guards (built into the script):
- rejects Content-Type other than text/html|application/xhtml+xml|text/plain|application/xml|text/xml → exit 4
- sniffs raw HTML for Cloudflare / "Please enable JavaScript" / Imperva / DataDome wall markers → exit 2
- rejects extracted bodies under 50 chars (configurable via --min-body N, 0 to disable) → exit 2

Read the full file on GitHub · 83 lines

Files

What ships with it

3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

1mo ago First seen · 83 lines · 90 tokens per session scan A 075bf66f381b

Subscribe to this mod's changes

fetch-url-as-markdown is a skill published in the GitHub repository CodeAlive-AI/ai-driven-development (163 stars, last pushed 2d ago), licensed MIT. It adds 90 tokens to every session and 953 once invoked, about $0.0004 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Browser automation Claude Code Research agent-skills ai-coding ai-driven-development antigravity codex-cli

webxr-3d-spatial-design

Technical guidelines for building immersive WebXR, Three.js, and WebGPU 3D spatial experiences. Use when implementing WebXR session management (VR/AR/inline), controller/hand tracking inputs, spatial audio, 3D UI raycasting, GLTF loading optimization, or WebGL shader performance.

hamzabellouch/agent-skills · 70 tokens

webrtc-realtime-media-streaming

Architect and deploy sub-second real-time video, audio, and data streaming using WebRTC, SFUs (Selective Forwarding Units), STUN/TURN traversal, SDP negotiation, and data channels. Trigger when implementing real-time video conferencing, live interactive streaming, or ultra-low latency peer-to-peer data synchronization.

hamzabellouch/agent-skills · 69 tokens

x-timeline

Read an authenticated X Following or For You timeline through agent-browser without APIs or engagement actions.

dceoy/ai-coding-agent-skills · 22 tokens

oracle-chatgpt

Run exactly one arbitrary prompt through ChatGPT via Oracle browser mode and return the captured response without reinterpretation. Use when the user explicitly wants a generic ChatGPT-via-Oracle consult without task-specific orchestration.

dceoy/ai-coding-agent-skills · 47 tokens

webapp-testing

Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.

hamzabellouch/agent-skills · 35 tokens

analyzing-tls-certificate-transparency-logs

Queries Certificate Transparency logs via crt.sh and pycrtsh to detect phishing domains, unauthorized certificate issuance, and shadow IT. Monitors newly issued certificates for typosquatting and brand impersonation using Levenshtein distance. Use for proactive phishing domain detection and certificate monitoring.

marysatasselshaped667/skills-collection-1 · 67 tokens