What is fetch-url-as-markdown?

A web-page reader that downloads a URL and returns the main content as clean Markdown, a plain-text format commonly used in code and documentation.

Use it to fetch, read, scrape, summarize, or quote web pages. It is not intended for PDFs, images, or archive files.

It strips away navigation, footers, and advertisements, and can use a fallback when a page needs JavaScript or blocks normal downloads.

fetch-url-as-markdown — Skill for Claude Code and Codex by CodeAlive-AI: A web-page reader that downloads a URL and returns…

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods

npx agentmods add skills/codealive-ai/ai-driven-development/fetch-url-as-markdown

Any agent

npx skills add CodeAlive-AI/ai-driven-development --skill fetch-url-as-markdown

Clone the repo

git clone --depth 1 https://github.com/CodeAlive-AI/ai-driven-development

Made for: Claude Code, Codex.

Or install ai-driven-development, the plugin that ships this one along with the rest of its 25 skills, 1 agent.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/skills/codealive-ai/ai-driven-development/fetch-url-as-markdown.svg)](https://agentmods.dev/skills/codealive-ai/ai-driven-development/fetch-url-as-markdown)

Your own site

<a href="https://agentmods.dev/skills/codealive-ai/ai-driven-development/fetch-url-as-markdown"><img src="https://agentmods.dev/badge/skills/codealive-ai/ai-driven-development/fetch-url-as-markdown.svg" alt="Measured on agentmods" height="20"></a>

Per session 90 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.

When invoked 953 The whole file, excluding the scripts and references it only reads on demand.

Security scan A 0 findings. Scan, not verified.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5	$0.00090	$0.00953
Opus 5	$0.00045	$0.00477
Sonnet 5	$0.00018	$0.00191
Haiku 4.5	$0.00009	$0.00095

Measured 4d ago against content hash 075bf66f381b, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

fetch-url-as-markdown scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.

The scan reads SKILL.md. This mod also ships 1 executable file (scripts/fetch_url.py), listed below but not scanned — reading those needs a real analyzer, not pattern matching.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/fetch-url-as-markdown/SKILL.md · 83 lines

How it starts

The opening of the file, as written. The whole thing — 83 lines — stays where its author put it; the contents beside it link to each section on GitHub.

URL to Markdown

Fetch any web URL and get clean, readable Markdown — main content only, no navigation/footer/ads. Local + free by default; smart fallback to Exa MCP when the page can't be extracted locally.

Workflow (the only thing the agent needs to remember)

Try trafilatura first:

python3 ~/.claude/skills/fetch-url-as-markdown/scripts/fetch_url.py "<URL>"

If exit code is 1 or 2 → fall back to Exa MCP with the same URL:
```
mcp__exa__web_search_advanced_exa(
    query="<URL>",
    includeDomains=["<host of URL>"],
    numResults=1,
    textMaxCharacters=50000,
    type="auto"
)
```
(mcp__exa__crawling works too if the server exposes it; the web_search_advanced_exa call above is the always-available variant — pin the host with includeDomains and use the URL itself as the query.)
Exit code 3 means trafilatura is not installed — install once:
```
python3 -m pip install --break-system-packages trafilatura
```

Exit codes (what they mean for the fallback decision)

Code	Meaning	Action
0	Markdown printed to stdout	done
1	DownloadError — network/HTTP/timeout/anti-bot block at fetch	fall back to Exa
2	ExtractionError — empty extract, JS/Cloudflare wall, or stub body (<200 chars)	fall back to Exa
3	trafilatura missing	install (see above), then retry
4	UnsupportedContentTypeError — URL is binary (PDF, image, archive)	don't fall back to Exa; use the right specialized skill (e.g. `pdf` for PDFs)

Defaults baked into the script

output_format="markdown", include_formatting=True — keeps headings/lists/code structure where the source HTML uses real <h1..h6> etc.
include_links=True, include_tables=True
with_metadata=True → emits a YAML frontmatter (title, author, date, url, hostname)
favor_recall=True, deduplicate=True — readable but trims duplicates
Real-browser User-Agent + 30s timeout configured in scripts/settings.cfg
Anti-stub guards (built into the script):
- rejects Content-Type other than text/html|application/xhtml+xml|text/plain|application/xml|text/xml → exit 4
- sniffs raw HTML for Cloudflare / "Please enable JavaScript" / Imperva / DataDome wall markers → exit 2
- rejects extracted bodies under 50 chars (configurable via --min-body N, 0 to disable) → exit 2

Read the full file on GitHub · 83 lines

Files

What ships with it

3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

4d ago First seen · 83 lines · 90 tokens per session scan A 075bf66f381b

Subscribe to this mod's changes

fetch-url-as-markdown is a skill published in the GitHub repository CodeAlive-AI/ai-driven-development (132 stars, last pushed today), licensed MIT. It adds 90 tokens to every session and 953 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Browser automation agent-skills ai-coding ai-driven-development antigravity claude-code codex-cli

oracle-chatgpt

Run exactly one arbitrary prompt through ChatGPT via Oracle browser mode and return the captured response without reinterpretation. Use when the user explicitly wants a generic ChatGPT-via-Oracle consult without task-specific orchestration.

dceoy/ai-coding-agent-skills · 47 tokens

x-timeline

Read an authenticated X Following or For You timeline through agent-browser without APIs or engagement actions.

dceoy/ai-coding-agent-skills · 22 tokens

antigravity-workflows-v2

Antigravity Workflows workflow skill. Use this skill when the user needs Orchestrate multiple Antigravity skills through guided workflows for SaaS MVP delivery, security audits, AI agent builds, and browser QA and the operator should preserve the upstream workflow, copied support files, and provenance before merging…

diegosouzapw/awesome-omni-skills · 71 tokens

fetch-url-as-markdown