mcp-server-tester AGENTS.md

Repository instructions for a TypeScript testing framework built with Playwright, a tool for automating web browsers, to test MCP servers. MCP is a standard way for AI tools to connect to external services.

In plain words
What is it for?
Use them when changing MCP client code, browser-based integration tests, evaluation datasets or the test-reporting interface.
Why use it?
They collect the commands and project structure needed to build, test, type-check and lint the framework without having to discover them manually.

Instructions file for CodexOpenCode

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add instructions/gleanwork/mcp-server-tester/agents-md
Clone the repo
git clone --depth 1 https://github.com/gleanwork/mcp-server-tester

Made for: Codex, OpenCode.

Per session 3,456 This file is loaded in full into every session.
When invoked 3,456 The same file — it is already loaded in full.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.03456 $0.03456
Opus 5 $0.01728 $0.01728
Sonnet 5 $0.00691 $0.00691
Haiku 4.5 $0.00346 $0.00346

Measured 3d ago against content hash e1db80eca8ec, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

mcp-server-tester AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 3d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

AGENTS.md · 319 lines

How it starts

The opening of the file, as written. The whole thing — 319 lines — stays where its author put it; the contents beside it link to each section on GitHub.

AGENTS.md

This file provides guidance to coding agents when working with code in this repository.

Project Overview

@gleanwork/mcp-server-tester is a Playwright-based testing and evaluation framework for Model Context Protocol (MCP) servers. It provides Playwright fixtures for automated testing and data-driven eval datasets with optional LLM-as-a-judge scoring.

Common Commands

# Build (includes UI reporter build)
npm run build

# Unit tests (Vitest)
npm test                    # Run all unit tests
npm run test:watch          # Watch mode
npm test -- src/mcp/clientFactory.test.ts  # Run single test file
npm test -- -t "creates client"            # Run tests matching pattern

# Integration tests (Playwright)
npm run test:playwright

# Code quality
npm run typecheck           # TypeScript validation
npm run lint                # ESLint
npm run lint:fix            # Auto-fix lint issues
npm run format              # Prettier formatting
npm run format:check        # Check formatting

Architecture

Core Modules (src/)

  • config/ - MCPConfig types and Zod validation for stdio/HTTP transports
  • mcp/ - Client factory (createMCPClientForConfig), fixtures (MCPFixtureApi), and response normalization
  • auth/ - OAuth 2.1 with PKCE (PlaywrightOAuthClientProvider) and static token utilities
  • assertions/ - Unified assertion architecture (see below)
  • evals/ - Dataset types, loader, and runner (uses validators internally)
  • judge/ - LLM-as-a-judge via Claude Agent SDK
  • spec/ - MCP protocol conformance checks
  • reporters/ - Custom Playwright reporter with React-based UI
  • cli/ - mcp-server-tester init and mcp-server-tester generate commands

Assertions Module (src/assertions/)

The assertion architecture provides a single API for both inline tests and data-driven evals:

  • validators/ - Pure validation functions: validateText, validateSchema, validatePattern, validateError, validateSize, validateResponse, validateToolCalls, validateToolCallCount
  • matchers/ - Playwright custom matchers (see table below)

Read the full file on GitHub · 319 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 3d ago First seen · 319 lines · 3,456 tokens per session scan A e1db80eca8ec

Subscribe to this mod's changes

mcp-server-tester AGENTS.md is an instructions file published in the GitHub repository gleanwork/mcp-server-tester (19 stars, last pushed 15d ago), licensed MIT. It adds 3,456 tokens to every session, about $0.0173 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.