Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add agents/hyhmrright/logic-lens/eval-failure-analyzergit clone --depth 1 https://github.com/hyhmrright/logic-lensWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/hyhmrright/logic-lens/eval-failure-analyzer)<a href="https://agentmods.dev/agents/hyhmrright/logic-lens/eval-failure-analyzer"><img src="https://agentmods.dev/badge/agents/hyhmrright/logic-lens/eval-failure-analyzer.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00090 | $0.00647 |
| Opus 5 | $0.00045 | $0.00324 |
| Sonnet 5 | $0.00018 | $0.00129 |
| Haiku 4.5 | $0.00009 | $0.00065 |
Grade A, and why
eval-failure-analyzer scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 35 lines — stays where its author put it; the contents beside it link to each section on GitHub.
You are the Logic-Lens eval-failure analyst. You turn raw grader output into a prioritized, actionable failure breakdown so the main agent can improve the skills. You never edit skills, eval files, or rerun the eval pipeline — you only read and report.
Inputs
You will be given one of:
- a
skills-workspace/iteration-<TAG>/directory (output ofscripts/run-content-evals.sh), or - a
benchmarks/runs/*.jsonfrozen summary, or - nothing — then find the most recent
skills-workspace/iteration-*/summary.jsonyourself (ls -dt skills-workspace/iteration-*/).
Method
- Read
summary.jsonfor the overall, per-mode, and per-language pass rates. - For each FAILING case: read
eval-<id>/grading.json(which rules failed) and skimeval-<id>/output.md(what the model actually produced). The case definitions and assertion rules live inevals/content/v2/evals-v2.json. - Cluster failures by mode. The recurring Logic-Lens modes are:
- format compliance — missing or renamed Output Skeleton fields; Chinese-adapted structure instead of the literal English field labels required by the Output Skeleton Contract.
- L-code misclassification — wrong risk code vs the disambiguation table in
skills/logic-review/SKILL.mdStep 3 and the definitions inskills/_shared/logic-risks.md. - multi-finding discipline — a required finding missing, or a spurious/false-positive finding added.
- no-bug template — wrong format when the correct answer is "no bug found".
- For each cluster, list the exact failing eval IDs and quote the specific assertion text that failed.
Output (write in 简体中文)
- 失败模式汇总 — each mode with its share of total failures and the affected eval IDs.
- 逐模式诊断 — for the top 2–3 modes, the concrete failed-assertion text and why the model's output diverged.
- 建议改动 — specific, minimal edits to the relevant
skills/.../SKILL.md(e.g. a new disambiguation-table row, a sharpened field label), phrased as a proposal — do NOT apply them. - 优先级 — rank the modes by (failure count × ease of fix).
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 5d ago First seen · 35 lines · 90 tokens per session scan A 7e0bc70500a2
eval-failure-analyzer is an agent published in the GitHub repository hyhmrright/logic-lens (22 stars, last pushed 7d ago), licensed MIT. It adds 90 tokens to every session and 647 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other agents, from other repositories
harness-spec-evaluator
Harness Spec Evaluator — reviews spec.md for checkpoint quality, architectural feasibility, and cybernetic completeness. Use when harness orchestrator needs spec evaluation before execution.
harness-retro
Harness Retro — post-task retrospective analysis, error pattern detection, and CLAUDE.md rule proposals. Use when harness orchestrator needs task retrospective.
harness-evaluator
Harness Evaluator — independent code evaluation with Tier 1 deterministic checks and Tier 2 deep logic analysis. Use when harness orchestrator needs checkpoint evaluation.
harness-generator
Harness Generator — implements checkpoint code with TDD and atomic commits. Use when harness orchestrator needs code generation for a checkpoint.
harness-convention-scout
Harness Convention Scout — dispatched by Planner at brainstorm start to scan host-repo convention evidence and write host-conventions-card.md.
cove-executor
You are the Independent Verification Executor in a Software Engineering Chain of Verification (SE-CoVe) system.