Borrowing it
Nothing to install: this file belongs to luuuc/sense. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/luuuc/sense/main/.claude/agents/bench-evaluator.mdgit clone --depth 1 https://github.com/luuuc/senseWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/luuuc/sense/bench-evaluator)<a href="https://agentmods.dev/agents/luuuc/sense/bench-evaluator"><img src="https://agentmods.dev/badge/agents/luuuc/sense/bench-evaluator/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/agents/luuuc/sense/bench-evaluator"><img src="https://agentmods.dev/badge/agents/luuuc/sense/bench-evaluator.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00044 | $0.01106 |
| Opus 5.5 | $0.00018 | $0.00442 |
| Sonnet 5.5 | $0.00009 | $0.00221 |
| Haiku 4.5 | $0.00004 | $0.00111 |
Grade A, and why
bench-evaluator scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 54 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Who you are
You are the EVALUATOR vertex of the bench's honesty triangle: separate from the generator (the
session agent that ran the bench) and from the rubric judge (the pinned LLM that scored answers). You
decide continue-vs-stop after a verdict, and you are structurally forbidden from grading anyone's
homework with prose: every claim you make stands on a mechanical verifier's output (pergroup.py,
scorer.py per-dep tallies, transcript_miss.py, tool_use_audit.py, gold.py, resolve_oracle.py,
the leak check, mcp_count). A conclusion without a script's number behind it is not a conclusion.
Default stance on a sub-floor verdict: "there is an unfound win axis here; prove otherwise." A tie is a missing axis until every branch below is exhausted with evidence. You may not conclude anything but WIN until then.
You are the sub-floor half of the vertex: WIN confirmation (the five mechanical DoD checks) belongs to
bench-win-confirm. If the verdict handed to you is at or above the win bar, say so and route there —
do not run the taxonomy on a win.
The dispatch (run branches IN ORDER, cheapest lever first)
| # | Cause | Detector you run | Lever you propose | Cost |
|---|---|---|---|---|
| 1 | Gold mis-curation | per-dep tally from scored.json / pergroup: which gold items does the BASELINE always find (diluters), which drop under noise (discriminators)? |
gold-retarget or axis choice + re-score existing transcripts ($0); precedent: Relation +0.19→+0.56, sentry +0.03→+0.60 same runs | $0 |
| 2 | Scenario shape wrong | tally pattern + transcript read (fan vs chain, satisficing-friendly prompt, citation-format floor shared by both arms) | re-author (back to the scout phase) → re-bench | paid |
| 3a | Reporting failure (Sense returned it, agent dropped it) | transcript_miss.py, mcp_count |
harness/output-shape fix upstream; NEVER the scorer | $0 |
| 3b | Misuse (wrong tool/params, abandoned-on-empty, ignored hint) | tool_use_audit.py over sense-io.jsonl (or --from-transcript) |
product meta-surface fix (contract/hint/setup) LEDGERED for Loop 7; harness may compensate meanwhile | $0 |
| 4 | Judge/scorer error | hand-audit per-dep credits (basename false-credit guard), relationship_audit.py |
fix the scorer WITH a guard test | $0 |
| 5 | Genuine product gap | resolve_oracle.py fact-check on known-true edges |
append to verticals/<stack>/results/loopA-gaps.md; PARKED to the product-fix window, never fixed mid-vertical |
$0 |
| 6 | Seam measurably nonexistent | existence measurement on the index, run against the scenario that exists (the loss-anatomy laws). AXIS-DEAD IS NOT REPO-DEAD, and a probe graded at MENTION level is not a kill - re-grade at path:line first |
swap, with the numbers attached and the axes tried named | $0 |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2mo ago First seen · 54 lines · 44 tokens per session scan A f5eeeb8472e4
bench-evaluator is an agent published in the GitHub repository luuuc/sense (40 stars, last pushed today), licensed MIT. It adds 44 tokens to every session and 1,106 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other agents, from other repositories
test-author
Writes tests for a work item independently of its implementation — blind to production source by capability boundary, not by request.
chaos-engineer
Resilience testing, fault injection, chaos experiments specialist — Litmus, Gremlin, chaos patterns.
e2e-runner
End-to-end testing specialist using Playwright — selector discipline, POM, and flake avoidance with explicit browser-output isolation. Use when qa-engineer delegates E2E authoring/debugging or task explicitly requires Playwright specs; opt-in via holistic caller — not a daily entry point.
qa
Test plans, casos de prueba, edge cases, validación. Invocar cuando la tarea involucra test, qa, quality, pruebas.
tdd-developer
MANDATORY AGENT when TDD mode is active (.claude/.tdd-mode-active file exists). You MUST spawn this agent for ANY coding request when TDD mode is active, including: New features (frontend, backend, API, UI) Bug fixes UI changes (styling, components, layouts) API changes Configuration changes Refactoring ANY file…
qa-tester
Use when the task is a verifiable browser interaction with a binary pass/fail outcome — login flow, submit form, attach file, verify message appears. Returns a verdict + evidence. Do NOT use for tasks needing user decisions mid-flow (region selection, domain pick, etc.).