Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
git clone --depth 1 https://github.com/Pierry/harness-kitWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/commands/pierry/harness-kit/eval)<a href="https://agentmods.dev/commands/pierry/harness-kit/eval"><img src="https://agentmods.dev/badge/commands/pierry/harness-kit/eval/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/commands/pierry/harness-kit/eval"><img src="https://agentmods.dev/badge/commands/pierry/harness-kit/eval.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00027 | $0.00566 |
| Opus 5.5 | $0.00011 | $0.00226 |
| Sonnet 5.5 | $0.00005 | $0.00113 |
| Haiku 4.5 | $0.00003 | $0.00057 |
Grade A, and why
eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 12d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
What it actually says
Set eval judge. State in .claude/hk-config.json, read by .claude/scripts/jev-judge.py.
No arg
Run python3 .claude/scripts/hk-config.py get eval, show line. Ask which judge: local or jev. Continue below.
local
python3 .claude/scripts/hk-config.py eval local. Done. Evals dispatch Claude evaluator, as default.
jev
python3 .claude/scripts/hk-config.py get eval. Ifkey=setalready, runpython3 .claude/scripts/hk-config.py eval jev --key-env {key_env}and stop.- Else ask user (AskUserQuestion), two options:
- Already in env var: ask var name (default
TYPESAFE_API_KEY). Runpython3 .claude/scripts/hk-config.py eval jev --key-env {NAME}. - Paste key: warn first, key pasted in chat lands in session transcript. Safer path: user runs it themselves with
!so stdin never echoes back:! printf '%s' 'tsk_...' | python3 .claude/scripts/hk-config.py eval jev --key -If user pastes key in chat anyway, pipe it the same way. Never echo key back, never write it anywhere except via script.
- Already in env var: ask var name (default
- Exit 4 = key still missing. Relay stderr line verbatim. Judge stays
jevbut evals fall back to Claude until key present. - Key source: https://console.typesafe.ai/keys. Paid API (input tokens only, ~$0.042/Mtok). Local only, never CI.
- Tell user restart Claude Code if key was just stored, so
envblock fromsettings.local.jsonloads.
What changes
Weighted-rubric evals (### Name (weight N%)) scored by Jev: each - check: one yes/no question, - absent: regex in code, one call per artifact. Failed checks become retry feedback. Jev unsure on checks that decide pass/fail → escalates to Claude evaluator. spec-satisfied + readiness gates stay Claude. Any Jev failure (no key, artifact over 32k tokens, API error) → exit 3 → Claude evaluator. Runs logged to .claude/runtime/outputs/evals/jev-judge.jsonl. Full flow: .claude/shared/pipeline-pattern.md section 4.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 12d ago First seen · 31 lines · 27 tokens per session scan A 92d46e3ace65
eval is a command published in the GitHub repository Pierry/harness-kit (4 stars, last pushed 12d ago), licensed MIT. It adds 27 tokens to every session and 566 once invoked, about $0.0001 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-10-05.
Other commands, from other repositories
test-plan
Generate a test strategy document — test matrix, coverage goals, and test pyramid allocation for the current project.
deps-update
Audit and update dependencies safely.
test-suite
Run comprehensive test suite with coverage analysis.
qa
Dispatch a verifiable browser task to the qa-tester subagent. Use for regression checks, smoke tests, and any task with a clean pass/fail outcome. Supports an EXHAUSTIVE mode that verifies every control behind an honesty gate.
test-gen
Generate comprehensive tests for a specified file with edge cases and error scenarios.
backprop
Trace a bug back to a spec gap and generate regression tests.