What is sitemcp CLAUDE.md?

Repository-specific instructions for sitemcp, an MCP server that crawls a website and makes its pages available to AI assistants. MCP is a standard way for AI tools to connect to external data and actions.

Use them to install, build, lint, type-check, test, run the server for a URL, list crawled pages, and retrieve page content.

They explain the commands, architecture, caching, and testing conventions needed to work on the project.

How ryoppippi/sitemcp configures Claude Code

Reuse

Borrowing it

Nothing to install: this file belongs to ryoppippi/sitemcp. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.

Copy the file

curl -O https://raw.githubusercontent.com/ryoppippi/sitemcp/main/CLAUDE.md

Clone the repo

git clone --depth 1 https://github.com/ryoppippi/sitemcp

Made for: Claude Code.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/instructions/ryoppippi/sitemcp/claude-md/github.svg)](https://agentmods.dev/instructions/ryoppippi/sitemcp/claude-md)

Your own site

<a href="https://agentmods.dev/instructions/ryoppippi/sitemcp/claude-md"><img src="https://agentmods.dev/badge/instructions/ryoppippi/sitemcp/claude-md/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/instructions/ryoppippi/sitemcp/claude-md"><img src="https://agentmods.dev/badge/instructions/ryoppippi/sitemcp/claude-md.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 535 This file is loaded in full into every session.

When invoked 535 The same file — it is already loaded in full.

Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.

Origin fork From a forked repository.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.00535	$0.00535
Opus 5.5	$0.00214	$0.00214
Sonnet 5.5	$0.00107	$0.00107
Haiku 4.5	$0.00053	$0.00053

Measured 2mo ago against content hash 2743001a391c, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

sitemcp CLAUDE.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

CLAUDE.md · 53 lines

How it starts

The opening of the file, as written. The whole thing — 53 lines — stays where its author put it; the contents beside it link to each section on GitHub.

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Commands

Development

bun install - Install dependencies
bun run build - Build the project using tsdown, outputs to dist/
bun run lint - Run Biome linter and formatter
bun run typecheck - Run TypeScript type checking using tsgo
bun test - Run unit tests
bun release - Full release process: lint, typecheck, test, build, and version bump

Running the MCP Server

bunx . [url] - Run the MCP server for a specific URL
Example: bunx . https://example.com --concurrency 5 --match "/**" --cache

Architecture

This is a Model Context Protocol (MCP) server that fetches websites and exposes them as tools for AI assistants.

Core Components

MCP Server (server.ts): Creates dynamic tools based on the fetched site:
- indexOf{SiteName} - Returns paginated list of all pages
- getDocumentOf{SiteName} - Returns content of specific pages
- Uses YAML format for responses
- Implements file-based caching in ~/.cache/sitemcp
Site Fetcher (fetch-site.ts): Queue-based concurrent web crawler:
- Uses Cheerio for link extraction
- Uses Happy DOM + Readability for content extraction
- Respects match patterns and same-origin policy
- Handles redirects and non-HTML content gracefully
CLI (cli.ts): Command-line interface using gunshi framework
- Validates inputs and configures the server
- Supports options for concurrency, caching, content selectors, etc.

Key Technical Decisions

Content Extraction: Uses Mozilla's Readability library to extract article content intelligently
Concurrent Fetching: Queue-based system with configurable concurrency (default: 3)
Tool Naming: Dynamic tool names generated from site URL (domain/subdomain/pathname strategies)
Response Format: YAML for better readability in AI contexts
Logging: Custom logger outputs to stderr (stdout reserved for MCP protocol)

Read the full file on GitHub · 53 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

2mo ago First seen · 53 lines · 535 tokens per session scan A 2743001a391c

Subscribe to this mod's changes

sitemcp CLAUDE.md is an instructions file published in the GitHub repository ryoppippi/sitemcp (761 stars, last pushed 6mo ago), licensed MIT. It adds 535 tokens to every session, about $0.0021 per session on Opus 5.5. A static security scan graded it A with 0 findings. It comes from a forked repository.

Browser automation

sitemcp CLAUDE.md next to 6 instructions from other repositories
Mod	Type	Stars	Last push	Licence	Scan	Tokens	Works with
sitemcp CLAUDE.mdryoppippi/sitemcp	Instructions file	761	6mo ago	MIT	A	535	Claude Code
vscode buildNext.instructions.mdmicrosoft/vscode	Instructions file	195k	yesterday	MIT	A	9,719	GitHub Copilot
next.js AGENTS.mdvercel/next.js	Instructions file	143k	yesterday	MIT	A	7,461	Codex, OpenCode
langchain AGENTS.mdlangchain-ai/langchain	Instructions file	149k	yesterday	MIT	A	4,680	Codex, OpenCode
vscode oss-third-party-notices.instructions.mdmicrosoft/vscode	Instructions file	195k	yesterday	MIT	A	5,516	GitHub Copilot
vscode copilot-instructions.mdmicrosoft/vscode	Instructions file	195k	yesterday	MIT	A	3,093	GitHub Copilot
deepseek-harness AGENTS.mddeepseek-ai/deepseek-harness	Instructions file	274k	2d ago	MIT	A	4,313	Codex, OpenCode

sitemcp: Instructions file for Claude Code