Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add gleanwork/mcp-server-tester --skill mcp-tester-guidegit clone --depth 1 https://github.com/gleanwork/mcp-server-testerWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/gleanwork/mcp-server-tester/mcp-tester-guide)<a href="https://agentmods.dev/skills/gleanwork/mcp-server-tester/mcp-tester-guide"><img src="https://agentmods.dev/badge/skills/gleanwork/mcp-server-tester/mcp-tester-guide/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/gleanwork/mcp-server-tester/mcp-tester-guide"><img src="https://agentmods.dev/badge/skills/gleanwork/mcp-server-tester/mcp-tester-guide.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 3 findings, up to medium
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- medium MCP Rug Pull · line 278 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 281 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 355 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00130 | $0.05527 |
| Opus 5.5 | $0.00052 | $0.02211 |
| Sonnet 5.5 | $0.00026 | $0.01105 |
| Haiku 4.5 | $0.00013 | $0.00553 |
Grade A, and why
mcp-tester-guide scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured today.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 320 lines — stays where its author put it; the contents beside it link to each section on GitHub.
MST reference
MST (@gleanwork/mcp-server-tester, CLI mst) tests and evaluates MCP servers. Requires Node.js 22 or later. 2.0 is in beta: npm install --save-dev @gleanwork/mcp-server-tester@beta @playwright/test.
- Tests check a server directly, with no client: a Playwright test calls a tool through the
mcpfixture and checks the result with MST's matchers. Deterministic; run once. Skill:write-mcp-test. - Evals have a client act on cases and grade what it did. A client (such as Claude Code) runs each case's input with a model; assertions and judges score each trial. Non-deterministic; run several trials. Skill:
write-mcp-eval. - Tool optimization runs tool-metadata variants against a baseline and recommends applying one or none. Skill:
optimize-mcp-tool-metadata.
Vocabulary
MST uses these terms in its API, config keys, CLI and reports. Use them, and not the old ones in the last column.
| Term | Meaning | Not |
|---|---|---|
| eval | What to evaluate and how: datasets, variants, graders, metrics | suite |
| eval config | The JSON file that defines one eval; mst run runs one |
manifest |
| dataset | A named list of cases | eval set |
| case | One input for a client to act on, with what is expected | scenario, test |
| input | The user's request, sent to the client as its prompt | scenario |
| expected | A case's ground truth: an answer, rubric criteria | golden |
| client | The MCP client application under test: mst, claude-code, codex, cowork, chatgpt |
host |
| model | The LLM the client runs, set beside the client | engine |
| variant | One setup an eval tests: client, model and options, servers, tool metadata | arm |
| baseline | The variant the others are compared with (the first, unless baseline names one) |
control |
| tool metadata | Tool names, descriptions and input schemas a variant shows the client (tools) |
tool overrides |
| trial | One attempt at a case by one variant | iteration |
| trace | What a client did in a trial: tool calls, skill loads, usage, final answer | transcript |
| grader | An assertion (code) or a judge (model) | scorer |
| assertion | A deterministic check of a trial's trace or answer (toolsTriggered, containsText, ...) |
expectation |
| judge | An LLM that scores a trial against expected or a rubric | evaluator |
| score | A grader's result for one trial: 0 to 1, pass, and why | verdict |
| pass threshold | The share of a case's trials that must pass (passThreshold, default 1) |
accuracy threshold |
| metric | An aggregate over a run's trials or cases: pass rate, tool count, tokens, cost | |
| run | One execution of an eval: every variant on every case | experiment |
| comparison | How a variant differs from the baseline, or a run from an earlier run | diff |
| test | A direct check of a server with MST's fixtures and matchers; not an eval | direct case |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- today Changed e3211f64657e
- 21d ago Changed · +2 lines 4250699a66d7
- 22d ago Changed · -174 lines · +57 tokens per session 66655d21a56c
- 23d ago Changed 843d957a602a
- 24d ago Changed 7ebfb11d556c
- 27d ago Changed · +12 lines 0d352baf0128
- 28d ago Changed bceffbff77d6
- 2mo ago First seen · 480 lines · 73 tokens per session scan A 77705f693eb7
mcp-tester-guide is a skill published in the GitHub repository gleanwork/mcp-server-tester (19 stars, last pushed today), licensed MIT. It adds 130 tokens to every session and 5,527 once invoked, about $0.0005 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
verify-omnigent
Drive Omnigent the way a user does and prove a behavior with recorded evidence, using an isolated server, runner, and mock model plus a feature map of every user entry point. Load before reproducing a user-facing bug, before claiming a fix works, or when reviewing whether a change covered every surface (session vs.…
test-master
Generates test files, creates mocking strategies, analyzes code coverage, designs test architectures, and produces test plans and defect reports across functional, performance, and security testing disciplines. Use when writing unit tests, integration tests, or E2E tests; creating test strategies or automation…
eval-dataset-design
A guide to creating reliable test sets for AI agents. Each evaluation task includes the request, the simulated user's behaviour, a reset starting state, and a success check that can be independently verified.
dogfood
Systematically explore and test a mobile app on iOS/Android with agent-device to find bugs, UX issues, and other problems. Use when asked to dogfood, QA, exploratory test, find issues, bug hunt, or test this app on mobile.
cua-driver
Use Cua Driver for desktop or browser tasks that are awkward or unavailable through Bash/APIs, or when the user explicitly wants GUI interaction: app testing, visual bug reproduction, form filling, calendar entry, screenshots, and demo recording. Also covers Cua setup; not OpenAI Codex Computer Use or web research.
test-warp-ui
Guides testing Warp UI features and changes using the computer use tool. Use this skill only when computer-use testing was requested (explicit request or accepted offer) and the computeruse tool is available to the agent. Covers launching Warp and verifying UI behavior.