Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add instructions/ryoppippi/sitemcp/claude-mdgit clone --depth 1 https://github.com/ryoppippi/sitemcpWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/instructions/ryoppippi/sitemcp/claude-md)<a href="https://agentmods.dev/instructions/ryoppippi/sitemcp/claude-md"><img src="https://agentmods.dev/badge/instructions/ryoppippi/sitemcp/claude-md.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00535 | $0.00535 |
| Opus 5 | $0.00267 | $0.00267 |
| Sonnet 5 | $0.00107 | $0.00107 |
| Haiku 4.5 | $0.00053 | $0.00053 |
Grade A, and why
sitemcp CLAUDE.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 6d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 53 lines — stays where its author put it; the contents beside it link to each section on GitHub.
CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
Commands
Development
bun install- Install dependenciesbun run build- Build the project using tsdown, outputs to dist/bun run lint- Run Biome linter and formatterbun run typecheck- Run TypeScript type checking using tsgobun test- Run unit testsbun release- Full release process: lint, typecheck, test, build, and version bump
Running the MCP Server
bunx . [url]- Run the MCP server for a specific URL- Example:
bunx . https://example.com --concurrency 5 --match "/**" --cache
Architecture
This is a Model Context Protocol (MCP) server that fetches websites and exposes them as tools for AI assistants.
Core Components
-
MCP Server (
server.ts): Creates dynamic tools based on the fetched site:indexOf{SiteName}- Returns paginated list of all pagesgetDocumentOf{SiteName}- Returns content of specific pages- Uses YAML format for responses
- Implements file-based caching in
~/.cache/sitemcp
-
Site Fetcher (
fetch-site.ts): Queue-based concurrent web crawler:- Uses Cheerio for link extraction
- Uses Happy DOM + Readability for content extraction
- Respects match patterns and same-origin policy
- Handles redirects and non-HTML content gracefully
-
CLI (
cli.ts): Command-line interface using gunshi framework- Validates inputs and configures the server
- Supports options for concurrency, caching, content selectors, etc.
Key Technical Decisions
- Content Extraction: Uses Mozilla's Readability library to extract article content intelligently
- Concurrent Fetching: Queue-based system with configurable concurrency (default: 3)
- Tool Naming: Dynamic tool names generated from site URL (domain/subdomain/pathname strategies)
- Response Format: YAML for better readability in AI contexts
- Logging: Custom logger outputs to stderr (stdout reserved for MCP protocol)
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 6d ago First seen · 53 lines · 535 tokens per session scan A 2743001a391c
sitemcp CLAUDE.md is an instructions file published in the GitHub repository ryoppippi/sitemcp (758 stars, last pushed 4mo ago), licensed MIT. It adds 535 tokens to every session, about $0.0027 per session on Opus 5. A static security scan graded it A with 0 findings. It comes from a forked repository.
Other instructions, from other repositories
Browser4 CLAUDE.md
Claude Code instructions for platonai/Browser4, covering browser4 — project context for claude, architecture, key dispatch chain (cli → browser), batch commands and e2e test structure.
cc-websearch CLAUDE.md
Claude Code instructions for Djarvur/cc-websearch, covering project, constraints, technology stack, what not to add and conventions.
fix-quera AGENTS.md
Instructions for AlirezaKeshavarz83/fix-quera, covering agent guidance, project shape, visual style, quera page data findings and compatibility findings.
sniff AGENTS.md
AGENTS.md instructions for Aboudjem/sniff, covering agents.md: sniff, what this repo is, how an agent should use sniff, handling the playwright setup gate and finding output schema.
fast-browser CLAUDE.md
Claude Code instructions for m4ttstack/fast-browser, covering fast browser plugin, where a change belongs, fork branch: use fast-browser-runtime, releasing a new runtime and re-pinning this repo: use the script.
sendblue-browser-use AGENTS.md
Instructions for sendblue-api/sendblue-browser-use, covering agents.md — sendblue-browser-use, what this repo is, setup, common commands and health (no auth).