testing-mcp-with-cli-agents

A testing workflow that drives real command-line coding agents to check an MCP server end to end. An MCP server provides tools that an agent can call.

In plain words
What is it for?
Use it to test MCP behavior with agents such as Claude, Codex, Cursor, Gemini, Grok, agy, or opencode in isolated tmux sessions.
Why use it?
It catches problems that unit tests may miss, such as tool discovery, approval prompts, cancellation, and unsafe test-session setup.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/tmux-python/libtmux-mcp/testing-mcp-with-cli-agents
Any agent
npx skills add tmux-python/libtmux-mcp --skill testing-mcp-with-cli-agents
Clone the repo
git clone --depth 1 https://github.com/tmux-python/libtmux-mcp

Made for: Claude Code, Codex.

Per session 181 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 3,999 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00181 $0.03999
Opus 5 $0.00090 $0.01999
Sonnet 5 $0.00036 $0.00800
Haiku 4.5 $0.00018 $0.00400

Measured 2d ago against content hash faa9a702a3c2, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

testing-mcp-with-cli-agents scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.agents/skills/testing-mcp-with-cli-agents/SKILL.md · 287 lines

How it starts

The opening of the file, as written. The whole thing — 287 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Testing an MCP server through real CLI agents

Unit tests prove the server's internals; they don't prove a real agent can discover a tool, get past its approval gate, call it, and survive cancelling it mid-flight. This skill exercises that whole path by pointing installed CLI agents at a checkout and driving them — with the tmux tool surface (libtmux-mcp) as the running example, though the shape generalizes to any MCP server.

The core idea: three tmux servers, never one

The single biggest mistake is running everything on one tmux socket. Keep three distinct servers, each on its own socket, and the whole thing becomes safe and observable:

Role Socket Who touches it Why separate
Your real session default you, interactively must never be mutated by a test
Harness tmux -L cli-harness the driver: send-keys prompts in, capture-pane render out isolates the CLI TUI you're driving
MCP-target tmux -L mcp-target, exported as LIBTMUX_SOCKET=mcp-target to the server the MCP server, when the agent calls tmux tools independent ground-truth; a destructive tool here can't kill the agent you're driving

The server resolves its socket from LIBTMUX_SOCKET and runs tmux -L <name> (src/libtmux_mcp/_utils.py builds argv with -L server.socket_name, defaulted from that env var). Exporting LIBTMUX_SOCKET=mcp-target into the server's config env fully sandboxes every tmux tool call onto a scratch server. Pre-create it so the agent has something to see:

$ tmux -L mcp-target new-session -d -s scratch

Keeping harness separate from MCP-target is the load-bearing part: if the agent's own TUI pane lived on the socket its MCP server mutates, one kill-server / kill-session tool call would tear down the agent mid-test, and your captures would be polluted by the agent's own UI redraws.

Climb only as high as the question needs — three fidelity layers

Layer 0 — Direct MCP smoke, no CLI at all

Read the full file on GitHub · 287 lines

Files

What ships with it

1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 287 lines · 181 tokens per session scan A faa9a702a3c2

Subscribe to this mod's changes

testing-mcp-with-cli-agents is a skill published in the GitHub repository tmux-python/libtmux-mcp (12 stars, last pushed 2d ago), licensed MIT. It adds 181 tokens to every session and 3,999 once invoked, about $0.0009 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

intuitive-port-worktree

Port changes from one git worktree or checkout into the default repository folder's current branch, then by default sync the result to the remote default branch when everything is clean. Use when the user asks to move, copy, transfer, transplant, cherry-pick, apply a patch, or port worktree changes into the…

MiaoDX/intuitive-flow · 79 tokens

comparative-synthesis

Compare and synthesize findings across multiple completed DeepScan reports. Use when the user wants cross-run analysis, trend comparison, or a unified summary from several research sessions.

papersflow-ai/papersflow-skills · 38 tokens

research-briefing

Build a focused literature and citation briefing from PapersFlow. Use when the user wants paper search, citation verification, related-paper discovery, or citation graph exploration.

papersflow-ai/papersflow-skills · 36 tokens

citation-verifier

Verify, normalize, and enrich a single citation or paper identifier. Use when the user pastes a DOI, URL, arXiv ID, PubMed ID, citation string, or paper title and wants it checked.

papersflow-ai/papersflow-skills · 48 tokens

deepscan-monitor

Run and monitor PapersFlow DeepScan jobs. Use when the user wants long-running research progress, intermediate findings, final reports, or plotting from a completed run.

papersflow-ai/papersflow-skills · 37 tokens

git-guardrails-claude-code

Set up Claude Code hooks to block dangerous git commands (push, reset --hard, clean, branch -D, etc.) before they execute. Use when user wants to prevent destructive git operations, add git safety hooks, or block git push/reset in Claude Code.

twaldin/flt · 62 tokens