Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add Ashfaqbs/software-dev-ai-claude-toolkit --skill eval-harnessgit clone --depth 1 https://github.com/Ashfaqbs/software-dev-ai-claude-toolkitWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/ashfaqbs/software-dev-ai-claude-toolkit/eval-harness)<a href="https://agentmods.dev/skills/ashfaqbs/software-dev-ai-claude-toolkit/eval-harness"><img src="https://agentmods.dev/badge/skills/ashfaqbs/software-dev-ai-claude-toolkit/eval-harness/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/ashfaqbs/software-dev-ai-claude-toolkit/eval-harness"><img src="https://agentmods.dev/badge/skills/ashfaqbs/software-dev-ai-claude-toolkit/eval-harness.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00019 | $0.01339 |
| Opus 5.5 | $0.00008 | $0.00536 |
| Sonnet 5.5 | $0.00004 | $0.00268 |
| Haiku 4.5 | $0.00002 | $0.00134 |
Grade A, and why
eval-harness scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
This is a copy
83% identical to eval-harness — 13 lines differ, which has more behind it and is treated as the original. This page carries a canonical link to it rather than competing with it.
How it starts
The opening of the file, as written. The whole thing — 228 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Eval Harness Skill
A formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.
Philosophy
Eval-Driven Development treats evals as the "unit tests of AI development":
- Define expected behavior BEFORE implementation
- Run evals continuously during development
- Track regressions with each change
- Use pass@k metrics for reliability measurement
Eval Types
Capability Evals
Test if Claude can do something it couldn't before:
[CAPABILITY EVAL: feature-name]
Task: Description of what Claude should accomplish
Success Criteria:
- [ ] Criterion 1
- [ ] Criterion 2
- [ ] Criterion 3
Expected Output: Description of expected result
Regression Evals
Ensure changes don't break existing functionality:
[REGRESSION EVAL: feature-name]
Baseline: SHA or checkpoint name
Tests:
- existing-test-1: PASS/FAIL
- existing-test-2: PASS/FAIL
- existing-test-3: PASS/FAIL
Result: X/Y passed (previously Y/Y)
Grader Types
1. Code-Based Grader
Deterministic checks using code:
# Check if file contains expected pattern
grep -q "export function handleAuth" src/auth.ts && echo "PASS" || echo "FAIL"
# Check if tests pass
npm test -- --testPathPattern="auth" && echo "PASS" || echo "FAIL"
# Check if build succeeds
npm run build && echo "PASS" || echo "FAIL"
2. Model-Based Grader
Use Claude to evaluate open-ended outputs:
[MODEL GRADER PROMPT]
Evaluate the following code change:
1. Does it solve the stated problem?
2. Is it well-structured?
3. Are edge cases handled?
4. Is error handling appropriate?
Score: 1-5 (1=poor, 5=excellent)
Reasoning: [explanation]
3. Human Grader
Flag for manual review:
[HUMAN REVIEW REQUIRED]
Change: Description of what changed
Reason: Why human review is needed
Risk Level: LOW/MEDIUM/HIGH
Metrics
pass@k
"At least one success in k attempts"
- pass@1: First attempt success rate
- pass@3: Success within 3 attempts
- Typical target: pass@3 > 90%
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 228 lines · 19 tokens per session scan A 4b2bb9f128ac
eval-harness is a skill published in the GitHub repository Ashfaqbs/software-dev-ai-claude-toolkit (27 stars, last pushed 1mo ago), licensed MIT. It adds 19 tokens to every session and 1,339 once invoked, about $0.0001 per session on Opus 5.5. A static security scan graded it A with 0 findings. It is 83% identical to eval-harness, differing in 13 lines, and is treated as a copy.
Other skills, from other repositories
execute
Run a plan step-by-step with verification gates, honoring the autonomy contract — never advances silently past a failure. Use when a drafted plan is ready to implement. User-invoked: /execute [slug | path] [--from N] [--strategy sequential|subtask|multi-agent]. Never auto-fires.
write-tests
Author a quality test suite for EXISTING code — convention-matching, test plan confirmed before any writing, pins current behavior but flags suspicious-as-bug. Use on /write-tests or "write / add tests for existing code". Mode A — never auto-fires.
close-out
Close out the scope chain on a shipped slug — leaves the canonical spec reconciled with what shipped, spent plan/blueprint artefacts archived, and one ship line recorded. The terminus after /execute meets done-criteria. Use on /close-out , "close out this slug", "reconcile what shipped", "wrap up ". Mode A — never…
test-driven-development
Use when implementing a non-trivial feature or bugfix where regressions matter, before writing implementation code. Skip for throwaway prototypes, generated code, pure config changes, or one-line fixes — but only with the user's permission.
playwright-cli
Automates browser interactions for web testing, form filling, screenshots, and data extraction using playwright-cli. Use when the user needs to navigate websites, interact with web pages, fill forms, take screenshots, test web applications, extract information from web pages, mock network requests, manage browser…
common-subagent-driven-development
Execute multi-task implementation plans by dispatching a fresh implementer subagent per task with independent review gates. Use when implementing plans, executing multi-task PRs, or transitioning from planning to implementation.