Borrowing it
Nothing to install: this file belongs to Purewhiter/mobilegym. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/Purewhiter/mobilegym/main/.claude/skills/testing-bench-task/SKILL.mdgit clone --depth 1 https://github.com/Purewhiter/mobilegymWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/purewhiter/mobilegym/testing-bench-task)<a href="https://agentmods.dev/skills/purewhiter/mobilegym/testing-bench-task"><img src="https://agentmods.dev/badge/skills/purewhiter/mobilegym/testing-bench-task/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/purewhiter/mobilegym/testing-bench-task"><img src="https://agentmods.dev/badge/skills/purewhiter/mobilegym/testing-bench-task.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00074 | $0.01676 |
| Opus 5.5 | $0.00030 | $0.00670 |
| Sonnet 5.5 | $0.00015 | $0.00335 |
| Haiku 4.5 | $0.00007 | $0.00168 |
Grade A, and why
testing-bench-task scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 122 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Testing bench_env Task Judges
Overview
A single positive + a single "opposite of positive" negative is not a test — it's a tautology. Real coverage means the negative exercises a failure mode an Agent could realistically produce. Most defects ship because tests stop at one reverse pattern.
Authoritative ref: bench_env/docs/task/TASK_TESTING_GUIDE.md §4 (read §4.3.2 through §4.3.5 before writing cases).
Rule 1 — Positive answer must be natural language (§4.3.2)
AnswerTask positives with bare ground-truth values (answer="32", answer="多云") bypass match_value's fuzzy/number-extraction logic — the test proves nothing.
# ❌ bare ground truth — test passes tautologically
_make_input(state, state, answer="多云")
# ✅ natural language, as an Agent would actually respond
_make_input(state, state, answer="上海今天天气多云转晴")
Format must include ground truth as substring / extractable number, plus realistic surrounding context.
Rule 2 — Pick negative patterns from the taxonomy (§4.3.3)
Blanket answer="错误答案" / curr_state == init_state is a single degenerate pattern. For each task, pick ≥1 pattern from the matching table:
Query tasks
| Pattern | Example |
|---|---|
| 查错对象 | Ask Beijing temp; answer Shanghai's real temp |
| 值接近但不对 | Ground truth 32; answer "北京现在33度" |
| 同义但语义不同 | GT="多云"; answer "今天阴天" |
| 过度回答含干扰数字 | GT humidity=40; answer "气温32度，紫外线7，风力3级" (no 40) |
| 布尔翻转 (肯/否定 subset) | GT=肯定; answer "没有通过核验" |
| 空回答 | answer=None or answer="" |
Operate tasks
| Pattern | Example |
|---|---|
| 未操作 | curr_state == init_state |
| 做反操作 | Target is close; state set to open |
| 操作错误目标 | Task wants tempUnit changed; curr changed windUnit |
| 部分完成 | 3-step sequential; only first step committed |
Cross-app tasks
| Pattern | Example |
|---|---|
| 源完成、目标未动 | 查了天气但未发微信 |
| 信息传递错误 | 发了微信但内容与天气不匹配 |
| 全部未动 | 所有 App 状态同 init |
Every negative case must name a pattern from the relevant table. Complex tasks (multi-field, cross-app) need ≥2 different patterns.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2mo ago First seen · 122 lines · 74 tokens per session scan A 0266c0edb9f0
testing-bench-task is a skill published in the GitHub repository Purewhiter/mobilegym (841 stars, last pushed 20d ago), licensed Apache-2.0. It adds 74 tokens to every session and 1,676 once invoked, about $0.0003 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
meter-benchmark
Measure per-stage success probability before iterating on CocoFlow templates. Supports $meter benchmark --flow .
belt
Operate the belt CLI to evaluate headless coding agents (Claude Code, Cursor, Codex, Gemini, and others) end to end. Use when the user asks to write or run eval scenarios, compare agents, score outputs with rules or LLM judges, register a new agent adapter, interpret reports or benchmark cards, or set up evals in CI.…
foundry-hosted-agent-validation
Step-by-step process for validating a Python Foundry hosted agent sample (under python/samples/04-hosting/foundry-hosted-agents/) end to end — running it locally (native runtime and azd ai agent run) and after deploying it to an Azure AI Foundry project with azd. Use this when asked to validate a hosted agent sample.
build-and-test
How to build and test .NET projects in the Agent Framework repository. Use this when verifying or testing changes.
skill-release-gate
Evaluate an Agent Skill bundle for structural integrity, trigger quality, artifact improvement, script correctness, safety, installed-tree integrity, and target-host portability before release.
typescript-testing
Select and run TypeScript SDK verification for packages, examples, type checks, linting, builds, Vitest suites, Effect v4 CLI tests, and runtime E2E tests. Use when adding tests, diagnosing TypeScript CI, choosing a focused test command, validating TypeScript package changes, or writing/porting CLI tests against…