Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add instructions/cosmocoder/mcp-web-docs/agents-mdgit clone --depth 1 https://github.com/cosmocoder/mcp-web-docsWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.04700 | $0.04700 |
| Opus 5 | $0.02350 | $0.02350 |
| Sonnet 5 | $0.00940 | $0.00940 |
| Haiku 4.5 | $0.00470 | $0.00470 |
Grade A, and why
mcp-web-docs AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 701 lines — stays where its author put it; the contents beside it link to each section on GitHub.
MCP Web Docs - Agent Guidelines
This document provides comprehensive guidance for AI coding agents working on this project. It covers architecture, conventions, testing patterns, and best practices.
Project Overview
MCP Web Docs is a self-hosted Model Context Protocol (MCP) server that crawls, indexes, and searches documentation from any website. It provides:
- Universal web crawling with Playwright/Crawlee
- Hybrid search (full-text + semantic/vector search)
- Authentication support for protected documentation sites
- Local embeddings generation using FastEmbed
- Persistent storage with LanceDB (vectors) and SQLite (metadata)
Tech Stack
- Runtime: Node.js >= 24, npm >= 11
- Language: TypeScript (ES2022, NodeNext modules)
- Testing: Vitest with coverage
- Linting: ESLint with TypeScript support
- Formatting: Prettier
- Key Dependencies:
@modelcontextprotocol/sdk- MCP server implementationcrawlee+playwright- Web crawling@lancedb/lancedb- Vector databasenode:sqlite- Metadata storage, via theSqliteDatabasewrapper insrc/storage/sqlite.tsfastembed- Local embedding generationzod- Schema validation
Architecture
┌─────────────────────────────────────────────────────────────┐
│ MCP Web Docs Server │
├─────────────────────────────────────────────────────────────┤
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ Crawler │ │ Processor │ │ Storage │ │
│ │ (Crawlee) │ │ (Markdown, │ │ ┌───────────────┐ │ │
│ │ │ │ text) │ │ │ LanceDB │ │ │
│ │ - Playwright│ │ │ │ │ (Vectors) │ │ │
│ │ - Auth │ │ - Chunking │ │ └───────────────┘ │ │
│ │ - Extractors│ │ - Metadata │ │ ┌───────────────┐ │ │
│ │ │ │ - Embedding │ │ │ SQLite │ │ │
│ └─────────────┘ └─────────────┘ │ │ (Metadata) │ │ │
│ │ └───────────────┘ │ │
└────────────────────────────────────┴─────────────────────┴──┘
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- yesterday First seen · 701 lines · 4,700 tokens per session scan A 7a004eab78ee
mcp-web-docs AGENTS.md is an instructions file published in the GitHub repository cosmocoder/mcp-web-docs (1 stars, last pushed 8d ago), licensed MIT. It adds 4,700 tokens to every session, about $0.0235 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other instructions, from other repositories
Browser4 CLAUDE.md
Instructions for platonai/Browser4, covering browser4 — project context for claude, architecture, key dispatch chain (cli → browser), batch commands and e2e test structure.
spring-ai-agentcore AGENTS.md
Instructions for spring-ai-community/spring-ai-agentcore, covering agents.md, project overview, architecture, key components and artifact store classes.
flyto-core CLAUDE.md
Instructions for flytohub/flyto-core, covering claude notes, cross-agent handoff and shared code intelligence.
deckforge AGENTS.md
Instructions for tph-kds/deckforge, covering deckforge agent entry point, code intelligence, read order, default routing and non-negotiable implementation rules.
fast-browser CLAUDE.md
Claude Code instructions for m4ttstack/fast-browser, covering fast browser plugin, where a change belongs, fork branch: use fast-browser-runtime, releasing a new runtime and re-pinning this repo: use the script.
agentoscope AGENTS.md
Instructions for rafaelcg/agentoscope, covering agentoscope — agent instructions, what this is, layout, commands and conventions.