llm-d-bench-mcp: Instructions file for Claude Code

llm-d-bench-mcp — standalone MCP server (stdio) for the llm-d-benchmarking-agent

The thin MCP adapter split out of the agent repo: it re-exposes the agent's tools + knowledge + workflow to other people's MCP clients (Claude Desktop, Claude Code, Cursor). Pure mechanism: it imports the engine (app.*) from the llm-d-benchmarking-agent checkout — the judgment ships as MCP resources/prompts/server-instructions sourced from the agent's knowledge/ (data, never duplicated here). Full design of record → DESIGN.md. Run it with python -m llm_d_bench_mcp (or the llm-d-bench-mcp console script).

Repo structure

llm-d-bench-mcp/
├─ llm_d_bench_mcp/
│  ├─ __init__.py       exports build_server, main
│  ├─ __main__.py       python -m llm_d_bench_mcp → main()
│  ├─ server.py         low-level Server: list_tools/call_tool (run_tool) + stdio loop + wires resources/prompts
│  ├─ adapters.py       per-connection adapters: one ToolContext per stdio connection + ApproveFn + EmitFn
│  └─ content.py        knowledge exposure: doc://knowledge/<stem> resources + workflow prompts + server INSTRUCTIONS
├─ tests/               pytest suite (hermetic; needs the agent repo importable — see below)
├─ scripts/install.sh   the one-command installer (curl-able); clones the agent repo at latest main
├─ README.md            user-facing docs (install, tools, prompts, security)
└─ DESIGN.md            the implementation spec / design of record

Non-negotiables

1. Thin code, thick agent — adapters + transport only, no decision logic. All judgment lives in the agent repo's knowledge/ and is exposed, never duplicated.
2. Reuse, don't fork — list_tools mirrors the agent's tool_definitions(); call_tool → run_tool → the shared dispatch(). Don't re-implement validation or handlers here.
3. The approval gate is re-homed, not removed (adapters.py) — never a silent auto-approve of a mutation; the re-homing mechanism → DESIGN.md §5.
4. Security deferred to local/stdio single-user (DESIGN.md §11) — acceptable only over stdio; revisit before any HTTP/shared transport.
5. SDK pin: mcp>=1.28,<2, low-level mcp.server.lowlevel.Server (not FastMCP, not the v2 on the SDK's main branch). camelCase inputSchema/mimeType on the types.* models.
6. The engine is a checkout, not a pip dep — app.* must import from a real llm-d-benchmarking-agent checkout (its knowledge/, command policy, and sibling repos are read from disk at runtime). The installer clones the agent repo at latest main and pip install -es it into the same venv as this package. Never vendor engine code here.