Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/caesiumy/ko-design-md/docs-crawlernpx skills add CaesiumY/ko-design-md --skill docs-crawlergit clone --depth 1 https://github.com/CaesiumY/ko-design-mdWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/caesiumy/ko-design-md/docs-crawler)<a href="https://agentmods.dev/skills/caesiumy/ko-design-md/docs-crawler"><img src="https://agentmods.dev/badge/skills/caesiumy/ko-design-md/docs-crawler.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00256 | $0.01483 |
| Opus 5 | $0.00128 | $0.00741 |
| Sonnet 5 | $0.00051 | $0.00297 |
| Haiku 4.5 | $0.00026 | $0.00148 |
Grade A, and why
docs-crawler scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 136 lines — stays where its author put it; the contents beside it link to each section on GitHub.
docs-crawler
Crawl a documentation website into a single Markdown corpus that an LLM can consume as context.
When to use
The user wants to capture a multi-page documentation, design-system, or
reference site as Markdown — to feed an LLM, archive it, or research a design
system. This skill is also invoked by the design-md skill's research phase.
Inputs
- Site URL (required) — the documentation site, e.g.
https://socarframe.socar.kr/. Any page on the site works; page discovery starts from the site'ssitemap.xml. - Output directory (optional) — where to write the corpus. Defaults to
./{host}-docs/in the current working directory.
If the user has not provided a URL, ask for one — do not guess.
Running the crawl
From the repository root:
pnpm crawl:docs <site-url> [--out <dir>] [--external-images]
Equivalent direct form:
pnpm exec tsx .claude/skills/docs-crawler/scripts/crawl.ts <site-url> [--out <dir>]
The engine discovers pages via sitemap.xml (falling back to same-origin
link-following), fetches each page, extracts the main content, and converts it
to Markdown. It prints per-page progress and a final summary line.
By default it also localizes images: every external image and inline base64
data: image is downloaded into crawl/images/ and the Markdown is rewritten
to relative paths, so the corpus is self-contained for renderers that can't
fetch external URLs (Claude Design, offline previews). Pass --external-images
to skip downloading and keep the original external URLs instead.
Output
These artifacts are written under the output directory:
crawl-corpus.md— every successful page merged into one document, with a table of contents and aSource:URL per page. This is the primary deliverable.crawl/pages/{NNN}-{slug}.md— one file per page, each withsource_url/title/methodfrontmatter.crawl/images/— downloaded image files (skipped with--external-images), referenced from the corpus and page files by relative paths.crawl/manifest.json— a per-URL audit log (status, fetch method, extracted character count, error reason).
What ships with it
16 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
- evals/evals.json 3.9 KB
- scripts/crawl.ts 11 KB runs code
- scripts/crawl/args.test.ts 1.8 KB runs code
- scripts/crawl/args.ts 2.7 KB runs code
- scripts/crawl/corpus.test.ts 2.8 KB runs code
- scripts/crawl/corpus.ts 2.6 KB runs code
- scripts/crawl/discover.test.ts 3.5 KB runs code
- scripts/crawl/discover.ts 7.4 KB runs code
- scripts/crawl/extract.test.ts 7.1 KB runs code
- scripts/crawl/extract.ts 4.7 KB runs code
- scripts/crawl/fetch-page.test.ts 1.3 KB runs code
- scripts/crawl/fetch-page.ts 5.3 KB runs code
- scripts/crawl/images.test.ts 8.2 KB runs code
- scripts/crawl/images.ts 9.6 KB runs code
- scripts/crawl/types.ts 1.8 KB runs code
- trigger-eval-queries.json 3.4 KB
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 5d ago First seen · 136 lines · 256 tokens per session scan A 4fe3b326cae1
docs-crawler is a skill published in the GitHub repository CaesiumY/ko-design-md (42 stars, last pushed yesterday), licensed MIT. It adds 256 tokens to every session and 1,483 once invoked, about $0.0013 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
design-engineering
Premium design engineering skill for agentic workflows — produces high-end, distinctive UI designs using DESIGN.md as the portable contract across Pencil MCP (in-IDE canvas), Figma MCP (team handoff + design tokens), and Google Stitch (vibe exploration + AI generation). Enforces anti-generic principles, WCAG 2.2 AA…
oma-hwp
Convert HWP / HWPX / HWPML files to Markdown using kordoc. Extracts text, headings, tables, lists, images, footnotes, and hyperlinks. Use for Korean word processor files (Hangul), government documents, and AI-ready data preparation.
oma-pdf
Convert PDF files to Markdown using opendataloader-pdf. Extracts text, tables, headings, lists, and images with correct reading order. Use for PDF parsing, PDF to Markdown conversion, document extraction, and AI-ready data preparation.
Convert PDF to Markdown using opendataloader-pdf — extracts text, tables, headings, and images with correct reading order.
book-serif-index
Create an archival book-reader design system with serif-led pages, mono index navigation, aged paper surfaces, margin notes, and a premium catalog frame.
ss-studio
Turn a product brief and optional references into three distinct creative directions, a human-selected StyleSeed interaction plan, generated image/video asset jobs, a working UI prototype, and a verified prototype-first showcase reel. Use for client concepts, app interaction exploration, trendy but coherent UI…