markdown-for-agents-mcp: Instructions file for Claude Code

CLAUDE.md

This file provides guidance to Claude Code when working with the markdown-for-agents-mcp codebase.

Purpose

An MCP (Model Context Protocol) server that fetches URLs with JavaScript rendering and converts them to clean, token-efficient markdown for AI agents.

Architecture

┌──────────────────────────────────────────────────────────────────┐
│                        MCP Server Layer                          │
│  ┌──────────┐  ┌────────────┐  ┌─────────────┐  ┌───────────┐   │
│  │ index.ts │  │ fetchUrl   │  │ fetchUrls   │  │ webSearch │   │
│  │ (entry)  │──│ (tool)     │──│ (tool)      │──│ (tool)    │   │
│  └──────────┘  └────────────┘  └─────────────┘  └───────────┘   │
└──────────────────────────────────────────────────────────────────┘
                             │
                             ▼
┌──────────────────────────────────────────────────────────────────┐
│                       Service Layer                              │
│  ┌───────────┐  ┌─────────────┐  ┌──────────────┐  ┌─────────┐  │
│  │ fetcher   │  │ converter   │  │ webSearch    │  │ config  │  │
│  │(Playwright│──│ (HTML→MD)   │  │ (DuckDuckGo) │  │ (Zod)   │  │
│  └───────────┘  └─────────────┘  └──────────────┘  └─────────┘  │
└──────────────────────────────────────────────────────────────────┘

File Structure

src/
├── index.ts             # MCP server entry point, tool dispatch
├── fetcher.ts           # Playwright-based URL fetcher with JS rendering
├── converter.ts         # HTML to markdown conversion
├── config.ts            # Zod-validated configuration from env vars
├── tools/
│   ├── fetchUrl.ts      # Single URL fetch tool
│   ├── fetchUrls.ts     # Batch URL fetch tool
│   └── webSearch.ts     # Web search tool wrapper
├── services/
│   └── webSearch.ts     # DuckDuckGo search + result fetching
└── utils/
    ├── cache.ts          # LRU cache with TTL
    ├── domainBlacklist.ts  # Domain/URL filtering (blocklist + allowlist modes)
    ├── errors.ts         # Custom error classes
    └── logger.ts         # Structured logging + metrics