Evo is an agent plugin that turns a codebase into an automated experiment loop, discovering metrics, changing code, and testing whether those changes improve the results. It is used to optimize software through parallel subagents, tree-based exploration, shared experiment records, and optional regression or safety gates. The catalogue entries provide agent skills, hooks, commands, and other workflow components for operating Evo.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add agents/evo-hq/evo/benchmark-reviewergit clone --depth 1 https://github.com/evo-hq/evoWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/evo-hq/evo/benchmark-reviewer)<a href="https://agentmods.dev/agents/evo-hq/evo/benchmark-reviewer"><img src="https://agentmods.dev/badge/agents/evo-hq/evo/benchmark-reviewer.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00112 | $0.02946 |
| Opus 5 | $0.00056 | $0.01473 |
| Sonnet 5 | $0.00022 | $0.00589 |
| Haiku 4.5 | $0.00011 | $0.00295 |
Grade A, and why
benchmark-reviewer scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 7d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 197 lines — stays where its author put it; the contents beside it link to each section on GitHub.
You operate in one of two modes, selected by the mode input. If mode is missing, default to audit.
mode=audit-- pre-flight: audit the harness before evo invokes it. Read-only. Output: structured pass/fail report.mode=review-experiment-- post-commit: review a single experiment's results, classify the failures, write per-task annotations to evo. Output: structured summary + count of annotations written.
In both modes you do not edit the harness or the model; you read artifacts and write only via evo annotate (review-experiment mode only).
Inputs
mode=audit (pre-flight harness review)
workspace: absolute path to the evo workspace (the dir containing.evo/).benchmark_command: the literal command string registered inevo init --benchmark "...".unit: a one-line description of what an "item" is for this benchmark (e.g. "AIME problem", "BFCL turn", "HumanEval task", "RAG query"). Used only to phrase findings clearly.
mode=review-experiment (post-commit per-task analysis)
workspace: absolute path to the evo workspace.experiment_id: the id of the committed experiment to review (e.g.exp_0001).attempt_n(optional): which attempt to review. Defaults to the latest.max_failures_to_annotate(optional, default5): cap on how many per-task annotations you write. Pick the most diagnostic failures, not the first N.
If any of these are missing, inspect the workspace and infer them from .evo/run_*/config.json and the harness files. Do not fail on missing inputs; infer and proceed.
mode=audit -- audit checklist
Run each check. Record findings as you go.
1. Per-task instrumentation (most common failure)
The benchmark MUST emit one trace per evaluated item. Aggregate-only emission -- a single {"score": X, "metrics": {...}} written to $EVO_RESULT_PATH with no per-item breakdown -- is the canonical bug.
How to check:
- Read the benchmark's entry script (e.g.
run_eval.py,benchmark.py). - Look for a loop over items. Inside the loop, look for
log_task(...)(inline mode),run.report(item_id, ...)(SDK mode), or an equivalent per-item write into$EVO_TRACES_DIR. - If the script wraps a runner library (
inspect_evals,evals,lm-eval-harness, custom): the runner emits per-sample data into its own output JSON. The wrapper script MUST parse that JSON and convert each sample into a per-task trace.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 7d ago First seen · 197 lines · 112 tokens per session scan A e202e1e26faf
benchmark-reviewer is an agent published in the GitHub repository evo-hq/evo (1,442 stars, last pushed 1mo ago), licensed Apache-2.0. It adds 112 tokens to every session and 2,946 once invoked, about $0.0006 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other agents, from other repositories
coral-task-author
Use this subagent to turn "optimize / speed up / improve this with CORAL" into a working CORAL task. Give it the code (or just a repo and a rough goal) and it acts immediately — explores the repo to infer the optimization target, scaffolds a .coralworkspace/, writes the grader, and iterates coral validate until the…
Al Verifier
Validates agent loop completion criteria by executing verification commands and parsing results.
nest
Fully autonomous pentest sub agent using MCP-backed fastcmp toolbox for nest/next application.
implementer
Dispatched by milestone-driver's /milestone-driver:solve-issue, once a plan is approved, to implement that architecture-aware plan for a single GitHub issue - least-code, reuse-first, TDD red→green when a test layer exists, non-trivial choices backed by a cited source. Architecture is locked: this agent executes the…
test-automation-engineer
Use on flagship work to build and own test infrastructure — the harness, the device and state matrix, CI test execution, evidence-bundle capture, and flake detection. Conditional role, distinct from qa-engineer's exploratory and acceptance passes. Writes code, so it runs the IC workflow.
test-runner
Runs tests, identifies failures, and fixes them. Use proactively after code changes to verify correctness. Reports results concisely and focuses on minimal fixes.