Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add JSK9999/ai-nexus --skill eval-harnessgit clone --depth 1 https://github.com/JSK9999/ai-nexusWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/jsk9999/ai-nexus/eval-harness)<a href="https://agentmods.dev/skills/jsk9999/ai-nexus/eval-harness"><img src="https://agentmods.dev/badge/skills/jsk9999/ai-nexus/eval-harness/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/jsk9999/ai-nexus/eval-harness"><img src="https://agentmods.dev/badge/skills/jsk9999/ai-nexus/eval-harness.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00018 | $0.01323 |
| Opus 5 | $0.00009 | $0.00661 |
| Sonnet 5 | $0.00004 | $0.00265 |
| Haiku 4.5 | $0.00002 | $0.00132 |
Grade A, and why
eval-harness scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 10d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
This is a copy
83% identical to eval-harness — 11 lines differ, which has more behind it and is treated as the original. This page carries a canonical link to it rather than competing with it.
How it starts
The opening of the file, as written. The whole thing — 227 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Eval Harness Skill
A formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.
Philosophy
Eval-Driven Development treats evals as the "unit tests of AI development":
- Define expected behavior BEFORE implementation
- Run evals continuously during development
- Track regressions with each change
- Use pass@k metrics for reliability measurement
Eval Types
Capability Evals
Test if Claude can do something it couldn't before:
[CAPABILITY EVAL: feature-name]
Task: Description of what Claude should accomplish
Success Criteria:
- [ ] Criterion 1
- [ ] Criterion 2
- [ ] Criterion 3
Expected Output: Description of expected result
Regression Evals
Ensure changes don't break existing functionality:
[REGRESSION EVAL: feature-name]
Baseline: SHA or checkpoint name
Tests:
- existing-test-1: PASS/FAIL
- existing-test-2: PASS/FAIL
- existing-test-3: PASS/FAIL
Result: X/Y passed (previously Y/Y)
Grader Types
1. Code-Based Grader
Deterministic checks using code:
# Check if file contains expected pattern
grep -q "export function handleAuth" src/auth.ts && echo "PASS" || echo "FAIL"
# Check if tests pass
npm test -- --testPathPattern="auth" && echo "PASS" || echo "FAIL"
# Check if build succeeds
npm run build && echo "PASS" || echo "FAIL"
2. Model-Based Grader
Use Claude to evaluate open-ended outputs:
[MODEL GRADER PROMPT]
Evaluate the following code change:
1. Does it solve the stated problem?
2. Is it well-structured?
3. Are edge cases handled?
4. Is error handling appropriate?
Score: 1-5 (1=poor, 5=excellent)
Reasoning: [explanation]
3. Human Grader
Flag for manual review:
[HUMAN REVIEW REQUIRED]
Change: Description of what changed
Reason: Why human review is needed
Risk Level: LOW/MEDIUM/HIGH
Metrics
pass@k
"At least one success in k attempts"
- pass@1: First attempt success rate
- pass@3: Success within 3 attempts
- Typical target: pass@3 > 90%
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 10d ago First seen · 227 lines · 18 tokens per session scan A 5078be09d63d
eval-harness is a skill published in the GitHub repository JSK9999/ai-nexus (19 stars, last pushed 6mo ago), licensed Apache-2.0. It adds 18 tokens to every session and 1,323 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. It is 83% identical to eval-harness, differing in 11 lines, and is treated as a copy.
Other skills, from other repositories
Cursor Skill (.mdc) Authoring
Author effective Cursor rules in .cursor/rules/.mdc - YAML frontmatter (description, globs, alwaysApply), the four rule types, scoped QA rules, subagents, and structure that actually steers the model.
test-fix
Diagnose and minimally fix a failing test, then re-run.
hatch3r-browser-verify
Opt-in browser verification skill — spec-run-first Playwright verification (assertions execute in the runner, agent reads only failures), axe-core a11y audits, toHaveScreenshot() regression diffs, E2E test scaffolds, and snapshot-mode exploratory driving. Default ON for UI-affecting agent invocations; disable globally…
hatch3r-feature
End-to-end feature implementation workflow. Covers data model, domain logic, API, and UI as a vertical slice. Use when implementing new features or working on feature request issues.
hatch3r-qa-path
Produces a human-run manual QA test path from a PR, branch diff, or uncommitted working tree — a risk-ordered table of steps, expected results, and automated-coverage references plus a shippability sign-off. Rows are auto-proven via delegated verification sub-agents before emission, so the human walks only what…
hatch3r-bug-fix
Step-by-step bug fix workflow. Diagnose root cause, implement minimal fix, write regression test. Use when fixing bugs, working on bug report issues, or when the user mentions a bug.