Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/hoja-solutions/agent-stdlib/build-agent-evalsnpx skills add Hoja-Solutions/agent-stdlib --skill build-agent-evalsgit clone --depth 1 https://github.com/Hoja-Solutions/agent-stdlibWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/hoja-solutions/agent-stdlib/build-agent-evals)<a href="https://agentmods.dev/skills/hoja-solutions/agent-stdlib/build-agent-evals"><img src="https://agentmods.dev/badge/skills/hoja-solutions/agent-stdlib/build-agent-evals.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00175 | $0.01353 |
| Opus 5 | $0.00088 | $0.00677 |
| Sonnet 5 | $0.00035 | $0.00271 |
| Haiku 4.5 | $0.00017 | $0.00135 |
Grade A, and why
build-agent-evals scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 80 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Build agent evals
Source: Demystifying evals for AI agents, with the section on keeping evals honest drawing on Eval awareness in BrowseComp and Designing AI-resistant technical evaluations. A standalone gist of the core material exists but is undiscoverable; this skill packages it and adds the runnable metric script.
An eval tells you whether a change to an agent made it better or worse. Without one you are guessing from vibes, and vibes miss regressions that only show up on the tenth run. Treat the eval suite the way you treat a unit-test suite: it has an owner, it grows when bugs slip through, and it fails loudly.
Start from real failures
Collect 20 to 50 tasks before writing any grader. The best sources are bugs your agent already produced, support tickets, and manual test cases you keep rerunning by hand. Write each task so two experts reading it reach the same verdict on pass or fail. If you cannot decide whether an output passed, the task is underspecified and will poison every measurement built on it.
Include a reference solution for each task to prove it is solvable, and build both positive cases (the agent should do X) and negative cases (the agent should refuse, or should not touch Y). A suite made only of positive cases optimizes toward an agent that does too much.
Choose the grader to match the task
Grade what the agent produced, not the path it took. An agent that reaches the right end state by an unusual route still passed.
- Code-based grader. String match, schema validation, a state check against a database or filesystem. Use this wherever the correct answer is checkable by a program. It is fast, free, and never flaky in the way a model judge is.
- Model-based grader (LLM-as-judge). A rubric scored by a separate model call. Use it for output that needs judgment: tone, summary quality, whether an explanation is correct. Give the judge a rubric with explicit criteria rather than asking "is this good," and have it cite evidence for its score so you can audit it.
- Human grader. Subject-matter spot checks and A/B preference. Use it sparingly to calibrate the other two, not as the everyday loop.
What ships with it
1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 4d ago First seen · 80 lines · 175 tokens per session scan A cc95e48254a9
build-agent-evals is a skill published in the GitHub repository Hoja-Solutions/agent-stdlib (1 stars, last pushed 1mo ago), licensed MIT. It adds 175 tokens to every session and 1,353 once invoked, about $0.0009 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
unified_meta_agent_skill
A foundational meta-agent skill for generating Multi-Agent Systems (MAS). It systematically drives the process from conceptual task decomposition to agent engineering, and finally to workflow orchestration.
controlflow-verify
Use after /controlflow-claude-code:controlflow-plan produces a plan, before implementation. Runs adversarial pre-execution verification inline in the main context (zero subagents): structural audit, assumption/mirage detection, and executability cold-start simulation. Emits APPROVED / NEEDSREVISION / REJECTED with…
controlflow-planning
Use when a repository task needs a strict ControlFlow-style implementation plan before coding, especially for small, medium, or large scope, cross-file edits, risky migrations, architectural uncertainty, or any work that should produce a saved Markdown plan artifact in plans/.
controlflow-router
Use when a task broadly matches ControlFlow for Codex and you need to decide whether to start with spec capture, strict workflow, strict planning, pre-execution plan review, orchestration, code review, or memory hygiene, or whether to combine several of those skills in sequence.
controlflow-plan
Use when a repository task needs a high-quality ControlFlow plan before coding — small, medium, or large scope, cross-file edits, risky migrations, architectural uncertainty, or any work that should produce a saved Markdown plan artifact in plans/. Single-sources the plan format from schemas/planner.plan.schema.json…
controlflow-spec
Use before planning non-trivial Codex work when the requirements, acceptance criteria, boundaries, constraints, or success measures are not yet clear enough for a safe implementation plan.