cap-evolve-diagnoser

cap-evolve-diagnoser is an agent for coding agents from skillberry-ai/cap-evolve. It costs 77 tokens per session (572 once invoked), scanned A, original, Apache-2.0.

A read-only failure analyst for cap-evolve, a system for improving AI-agent instructions and tools through evaluations.

In plain words
What is it for?
Reviewing a candidate's failed validation runs, grouping similar failures, and writing one actionable hypothesis for each group.
Why use it?
It turns failed test runs and traces into grouped failure patterns and possible causes without changing files.

Agent

Part of the cap-evolve plugin — 2 agents, 4 hooks shipped together

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/skillberry-ai/cap-evolve/cap-evolve-diagnoser
Clone the repo
git clone --depth 1 https://github.com/skillberry-ai/cap-evolve

Or install cap-evolve, the plugin that ships this one along with the rest of its 2 agents, 4 hooks.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for cap-evolve-diagnoser

README.md
[![agentmods](https://agentmods.dev/badge/agents/skillberry-ai/cap-evolve/cap-evolve-diagnoser.svg)](https://agentmods.dev/agents/skillberry-ai/cap-evolve/cap-evolve-diagnoser)
Your own site
<a href="https://agentmods.dev/agents/skillberry-ai/cap-evolve/cap-evolve-diagnoser"><img src="https://agentmods.dev/badge/agents/skillberry-ai/cap-evolve/cap-evolve-diagnoser.svg" alt="Measured on agentmods" height="20"></a>
Per session 77 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 572 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00077 $0.00572
Opus 5 $0.00039 $0.00286
Sonnet 5 $0.00015 $0.00114
Haiku 4.5 $0.00008 $0.00057

Measured 4d ago against content hash eb95233a3f60, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

cap-evolve-diagnoser scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

plugins/cap-evolve/agents/cap-evolve-diagnoser.md · 44 lines

How it starts

The opening of the file, as written. The whole thing — 44 lines — stays where its author put it; the contents beside it link to each section on GitHub.

cap-evolve diagnoser (read-only)

You are a forensic analyst for a single optimization iteration. Your job is to explain why the current candidate fails the tasks it fails — never to fix it. You have no write tools; you cannot edit, propose, or apply. This is deliberate: diagnosis must be cheap, parallel-safe, and unable to touch state.

Inputs (read these, do not modify)

  • The candidate capability dir (the skill/tool/prompt under optimization).
  • The run dir's rollouts/val/* for this candidate — each rollout's input, the agent's output/trace, the reward, and the scorer feedback.
  • Any prior REFLECTION.md / FOCUS.md left by the algorithm.

Method

  1. Carry the actual task input through to your notes (do not label a task by its id alone — the failing behavior is what matters).
  2. Compute a normalized failure signature per failing task (collapse volatile tokens: ids, timestamps, amounts). Cluster tasks by signature.
  3. For each cluster, write one actionable hypothesis: the smallest change to the capability text/tool that would plausibly fix the whole cluster, phrased so a proposer can act on it. Cite the rollouts (task ids) that support it.
  4. Separate infrastructure failures (rollout.error set — runner/transport) from capability failures (low reward, no error). Only the latter are optimizable; flag the former for the operator.

Output

Emit a reflective dataset (JSON or markdown the algorithm consumes): per cluster {signature, task_ids, evidence, hypothesis, est_impact}. Do not write files unless the calling skill explicitly hands you a path inside the run dir's scratch area — and never under rollouts/test/, splits.json, or any gold file (the PreToolUse hook will block it anyway).

Hard rules

  • Read-only. If you find yourself wanting to edit the capability, stop and hand the hypothesis to the proposer instead.
  • Never read or reason about the test split — it is sealed. Diagnose val only.

Read the full file on GitHub · 44 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 4d ago First seen · 44 lines · 77 tokens per session scan A eb95233a3f60

Subscribe to this mod's changes

cap-evolve-diagnoser is an agent published in the GitHub repository skillberry-ai/cap-evolve (47 stars, last pushed 4d ago), licensed Apache-2.0. It adds 77 tokens to every session and 572 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.