Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add gleanwork/mcp-server-tester --skill write-mcp-evalgit clone --depth 1 https://github.com/gleanwork/mcp-server-testerWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/gleanwork/mcp-server-tester/write-mcp-eval)<a href="https://agentmods.dev/skills/gleanwork/mcp-server-tester/write-mcp-eval"><img src="https://agentmods.dev/badge/skills/gleanwork/mcp-server-tester/write-mcp-eval/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/gleanwork/mcp-server-tester/write-mcp-eval"><img src="https://agentmods.dev/badge/skills/gleanwork/mcp-server-tester/write-mcp-eval.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 1 finding, up to medium
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- medium MCP Rug Pull · line 517 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00130 | $0.05391 |
| Opus 5.5 | $0.00052 | $0.02156 |
| Sonnet 5.5 | $0.00026 | $0.01078 |
| Haiku 4.5 | $0.00013 | $0.00539 |
Grade A, and why
write-mcp-eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured today.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 465 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Write MCP evals
An eval has a client act on cases and grades what it did. Each case gives the client an input (the user's request). The client runs it with a model and returns a trace: the tools it called and its final answer. Assertions and judges grade each trial, and MST decides whether the case passed.
A check that calls a tool with fixed arguments has no client, so it is a test, not an eval. Write those with the write-mcp-test skill. Datasets hold only cases for a client.
Before you start
- Decide what the eval measures: which tool the model picks, the arguments it passes, the quality of its answer, or cost and latency.
- Read the server's tools: names, descriptions and input schemas (
await mcp.listTools()in a test, or the server source). Every tool name in an assertion must exist. - Choose where the cases run:
Playwright test with runEvalDataset |
mst run with an eval config |
|
|---|---|---|
| Client | mst only, on the test's MCP connection |
mst, claude-code, codex, cowork, chatgpt, or a plugin's client |
| Compares | one setup per call | several variants in one run, each compared with the baseline |
| Results | the MCP Playwright reporter | a run directory with its report, and a table per variant |
Start with a Playwright test on mst while you write cases. Use an eval config to run them on another client or to compare variants.
Step 1: Write the dataset
A dataset is a JSON file with a name and a list of cases:
{
"name": "search-evals",
"cases": [
{
"id": "find-planning-doc",
"input": "Find the Q3 planning document",
"assertions": {
"toolsTriggered": {
"calls": [{ "name": "search", "required": true }]
}
}
}
]
}
What ships with it
1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- today Changed c81d92210db8
- 21d ago Changed · +2 lines 80561cda33ed
- 22d ago Changed · -55 lines · +74 tokens per session 627692b3f84b
- 23d ago Changed · +4 lines f099882aeec5
- 27d ago Changed fe1c6d8015be
- 28d ago Changed · -4 lines 239e116c2fac
- 2mo ago First seen · 518 lines · 56 tokens per session scan A 26737f27a335
write-mcp-eval is a skill published in the GitHub repository gleanwork/mcp-server-tester (19 stars, last pushed today), licensed MIT. It adds 130 tokens to every session and 5,391 once invoked, about $0.0005 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
verify-omnigent
Drive Omnigent the way a user does and prove a behavior with recorded evidence, using an isolated server, runner, and mock model plus a feature map of every user entry point. Load before reproducing a user-facing bug, before claiming a fix works, or when reviewing whether a change covered every surface (session vs.…
test-master
Generates test files, creates mocking strategies, analyzes code coverage, designs test architectures, and produces test plans and defect reports across functional, performance, and security testing disciplines. Use when writing unit tests, integration tests, or E2E tests; creating test strategies or automation…
eval-dataset-design
A guide to creating reliable test sets for AI agents. Each evaluation task includes the request, the simulated user's behaviour, a reset starting state, and a success check that can be independently verified.
dogfood
Systematically explore and test a mobile app on iOS/Android with agent-device to find bugs, UX issues, and other problems. Use when asked to dogfood, QA, exploratory test, find issues, bug hunt, or test this app on mobile.
cua-driver
Use Cua Driver for desktop or browser tasks that are awkward or unavailable through Bash/APIs, or when the user explicitly wants GUI interaction: app testing, visual bug reproduction, form filling, calendar entry, screenshots, and demo recording. Also covers Cua setup; not OpenAI Codex Computer Use or web research.
test-warp-ui
Guides testing Warp UI features and changes using the computer use tool. Use this skill only when computer-use testing was requested (explicit request or accepted offer) and the computeruse tool is available to the agent. Covers launching Warp and verifying UI behavior.