Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/haabe/mycelium/eval-runnernpx skills add haabe/mycelium --skill eval-runnergit clone --depth 1 https://github.com/haabe/myceliumWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00029 | $0.01061 |
| Opus 5 | $0.00015 | $0.00531 |
| Sonnet 5 | $0.00006 | $0.00212 |
| Haiku 4.5 | $0.00003 | $0.00106 |
Grade A, and why
eval-runner scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 92 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Eval Runner
Benchmark the agent's performance against defined scenarios. Adapted from n-trax eval system.
Commands
run <category/name>
- Read YAML from
.claude/evals/scenarios/<category>/<name>.yml - Parse fields (name, category, task_prompt, success_criteria, budget)
- Execute setup steps if defined
- Record start time
- Execute task via reflexion workflow (read corrections first)
- Record end time and iteration count
- Validate ALL success criteria
- Write result JSON to
.claude/evals/results/<timestamp>-<name>.json - Report summary
run-all [category]
- Glob
.claude/evals/scenarios/**/*.yml - Skip scenarios with
status: retired - For each: run in isolation (git stash), record result, restore
- Update
.claude/evals/pass-history.jsonwith each result - Aggregate and report
run-split <optimization|holdout>
- Glob
.claude/evals/scenarios/**/*.yml - Read each YAML, filter by
splitfield matching the requested set - Skip scenarios with
status: retired - For each matching scenario: run in isolation, record result, restore
- Update
.claude/evals/pass-history.jsonwith each result - Aggregate and report (label output clearly as "Optimization Set" or "Holdout Set")
report
- Read all results from
.claude/evals/results/ - Generate summary table:
| Category | Pass Rate | Avg Iterations | Avg Time | Notes |
|-------------|-----------|----------------|----------|-------|
| discovery | ... | ... | ... | |
| delivery | ... | ... | ... | |
| integration | ... | ... | ... | |
| **Overall** | ... | ... | ... | |
- List failure patterns and recommendations
prune
- Read
.claude/evals/pass-history.json - Flag evals where
last_5is all-pass (saturated) or all-fail (broken) - Flag evals with no runs in 30+ days (
stale) - For saturated evals, suggest: retire or increase difficulty
- For broken evals, suggest: fix criteria or retire
- Present recommendations — do NOT auto-retire
- On user confirmation: set
status: retiredin scenario YAML, update pass-history.json, log in .claude/harness/decision-log.md
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 92 lines · 29 tokens per session scan A f398c17e0461
eval-runner is a skill published in the GitHub repository haabe/mycelium (45 stars, last pushed 2d ago), licensed MIT. It adds 29 tokens to every session and 1,061 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
context-manager
Context management skill providing discovery, fetching, harvesting, extraction, compression, organization, cleanup, and guided workflows for project context.
prd-v05-technical-stack-selection
Determine technologies needed to build the product, making build/buy/integrate decisions during PRD v0.5 Red Team Review. Handles both greenfield and brownfield contexts. Triggers on requests to select tech stack, evaluate technologies, make build vs. buy decisions, discover existing assets, or when user asks "what…
prd-v06-architecture-design
Define how system components connect, establishing boundaries, patterns, and integration approaches during PRD v0.6 Architecture. Triggers on requests to design architecture, create system design, define component relationships, or when user asks "design architecture", "system design", "how do components connect?"…
prd-v08-monitoring-setup
Define monitoring strategy, metrics collection, and alerting thresholds during PRD v0.8 Deployment & Ops. Triggers on requests to set up monitoring, define alerts, or when user asks "what should we monitor?", "alerting strategy", "observability", "metrics", "SLOs", "dashboards", "monitoring setup". Outputs MON…
prd-v08-runbook-creation
Create operational playbooks for incident response, deployments, and maintenance during PRD v0.8 Deployment & Ops. Triggers on requests to create runbooks, document procedures, or when user asks "how do we handle incidents?", "runbook", "operational procedures", "on-call guide", "incident response", "maintenance…
prd-v09-feedback-loop-setup
Establish channels and processes for capturing and processing post-launch feedback during PRD v0.9 Go-to-Market. Triggers on requests to set up feedback systems, capture user input, or when user asks "how do we collect feedback?", "feedback loop", "user research", "post-launch feedback", "customer feedback", "NPS"…