Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/bytestack-labs/claude-plugins/tool-evalnpx skills add ByteStack-Labs/claude-plugins --skill tool-evalgit clone --depth 1 https://github.com/ByteStack-Labs/claude-pluginsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/bytestack-labs/claude-plugins/tool-eval)<a href="https://agentmods.dev/skills/bytestack-labs/claude-plugins/tool-eval"><img src="https://agentmods.dev/badge/skills/bytestack-labs/claude-plugins/tool-eval.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00253 | $0.01698 |
| Opus 5 | $0.00127 | $0.00849 |
| Sonnet 5 | $0.00051 | $0.00340 |
| Haiku 4.5 | $0.00025 | $0.00170 |
Grade A, and why
tool-eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 3d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 141 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Tool Eval
A tool is the contract between a deterministic system and a non-deterministic agent. When you score that contract with exact string match against a stored ground truth, the score itself becomes untrustworthy in two directions at once: it marks a correct answer wrong when the formatting differs, and it marks a wrong answer right when the answer happens to match a ground truth that is itself wrong. The headline number hides both.
This skill does not grade the tool with another model's opinion. It re-derives the tool's real accuracy from the raw inputs, separates the formatting misses from the genuine failures, and hands back a receipt anyone can rerun. The qualitative judge layer, whether the tool is well named and well documented, is kept, but it stays labeled as an opinion and never enters the number.
How to run it
Run tool-eval on this tool evaluation. Take the committed run transcript (tool calls, answers, stored ground truth) and score it three ways: naive exact match, normalized value match (strip currency, commas, whitespace, then compare the numbers), and a recompute from the raw inputs that ignores the stored ground truth. Report the three accuracies, the format-only mismatches (correct value, scored wrong), and the silent-wrong cases (scored right, value wrong). Keep any tool-quality feedback as a separate judge layer. Verify every number with runnable code and write a receipt that fails closed.
Core commitments
- Verified numbers only. Every figure traces to a measurement produced by runnable code. No remembered or approximate values.
- Score value, not string. Exact match is the scorer that created the false negative. The object of study is whether the tool returned the right value, measured against the answer recomputed from the raw inputs, not against a stored string.
- Recompute the ground truth. A stored ground truth can be wrong. Where the task allows the answer to be recomputed from its inputs, do that, and treat any disagreement with the stored truth as a finding, not noise.
- Judge and receipt are different layers. A model's opinion on tool quality is useful commentary. It is never mixed into the reproducible accuracy.
- Reproducibility is part of the deliverable. The fixture regenerates from a fixed, model-free transcript. Report the compute envelope (here, none: standard library).
- Never regenerate committed fixture data. If the target already ships a committed transcript, treat it as immutable and re-derive against it as-is.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 3d ago First seen · 141 lines · 253 tokens per session scan A 14cd64a89f33
tool-eval is a skill published in the GitHub repository ByteStack-Labs/claude-plugins (2 stars, last pushed 2mo ago), licensed MIT. It adds 253 tokens to every session and 1,698 once invoked, about $0.0013 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
writing-e2e-flows
Use when a feature or fix in future-agi needs an end-to-end Playwright flow under e2e/ — a new flow for user-visible behaviour, an update to a flow whose pinned endpoint, route, table or selector changed, or when a review flagged missing E2E coverage. Also use to check whether an existing flow already pins an…
reviewing-prs
Use when asked to review a pull request, branch, or diff — before approving, as a self-review before opening a PR, to judge whether a PR is ready — 'is PR 123 mergeable?', 'anything blocking here?' — or to answer 'does this change need an E2E flow'. Applies the FutureAGI coding standards and, in repos with an e2e/…
durable-executor
Universal first-hop for any tool-backed request that may execute, inspect, mutate, automate, browse, call, fetch, generate, install, test, run, or create artifacts through CLI, shell, API, adapter, browser, provider, local process, or external tool, with trace, alignment, evidence capture, and future recall. Use for…
generic-skill-creator
Generic example for creating or updating a Codex skill with appropriate resources, metadata, validation, and forward-testing. Use for generic skill creator, create a skill, update a skill, skill creator, skill authoring, SKILL.md, agents/openai.yaml, initskill.py, quickvalidate.py, forward-test a skill, progressive…
mcp-builder
Guide construction of MCP servers that expose external services through well-designed, documented, testable tools. Use for mcp-builder, MCP server, Model Context Protocol, build MCP, FastMCP, TypeScript MCP SDK, MCP SDK, MCP tools, MCP evaluations, MCP Inspector, streamable HTTP and stdio transport. Use when the task…
code-review
Multi-agent code review with deep analysis. Orchestrates codebase research, optional web research, parallel Rust engineers, codex second opinion, and general-purpose reviewers into a synthesized report. Use when the user asks to review code, review a PR, review changes, audit code quality, or says "review", "/review"…