Getting it into your agent
It runs from inside its repository, so the clone comes first — what it calls does not travel with the file alone.
git clone --depth 1 https://github.com/vasuag09/harness-claudenpx agentmods add skills/vasuag09/harness-claude/evalWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/vasuag09/harness-claude/eval)<a href="https://agentmods.dev/skills/vasuag09/harness-claude/eval"><img src="https://agentmods.dev/badge/skills/vasuag09/harness-claude/eval.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00054 | $0.00764 |
| Opus 5 | $0.00027 | $0.00382 |
| Sonnet 5 | $0.00011 | $0.00153 |
| Haiku 4.5 | $0.00005 | $0.00076 |
Grade A, and why
eval scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 6d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Runs shell commandslowCapability
Expected in a hook, worth knowing in a rule or an instructions file.
- **` ```eval ` fences are shell-executed** (via `_lib.sh` → `execSync`) with your privileges. How it starts
The opening of the file, as written. The whole thing — 54 lines — stays where its author put it; the contents beside it link to each section on GitHub.
/eval — acceptance-criteria checkpoint
Goal: prove the work satisfies the spec it was built against, criterion by criterion. Green tests show the code runs; this shows it meets what was promised.
Opt-in. This skill is not invoked by any default-pipeline skill or hook. Running it is always explicit; it changes no baseline behavior. (Fulfills v0.4 AC-6 / v0.3 AC-E5.)
Do this
- Locate the spec. Default to the latest
docs/specs/*.md; let the user name one if the change targets an older spec. Confirm it's the spec this change was built against. - Run the deterministic gate:
It prints a per-criterion table (PASS / FAIL / MANUAL) and exits 0 (all PASS, none manual) · 1 (a check FAILED) · 2 (no failures, but MANUAL criteria remain).node scripts/eval/checkpoint.js [spec.md]- Each criterion may carry an inline
```evalfenced shell command — that's its machine-check. Add one to an AC when a deterministic check exists (port it from the spec's own test commands).
- Each criterion may carry an inline
- On FAIL (exit 1): the criterion is objectively unmet. Fix the implementation (not the criterion) and re-run. Do not proceed while any check FAILs.
- Adjudicate MANUAL (exit 2): for each prose criterion the script can't decide, gather
concrete evidence (run the app, read the diff, check
.claude/traces/for what fired) and judge PASS or FAIL yourself. The script never auto-passes prose — closing these is your job. If a MANUAL criterion is genuinely verifiable, prefer adding an```evalfence so it's deterministic next time. - Gate. Ship only when every criterion is PASS — checks green and every MANUAL adjudicated PASS with evidence. Report the verdict criterion by criterion.
Security & trust
```evalfences are shell-executed (via_lib.sh→execSync) with your privileges. They are author-authored, committed markdown — trust them exactly like the repo's own test scripts or build hooks. Review specs from untrusted branches/PRs before running/eval.checkpoint.js --listis the safe inspect-before-run path: it parses and shows which ACs carry a check, but executes nothing.- Secret-safe: a check's output is never printed. On FAIL the report shows only the exit code + the command (committed text), so you re-run it yourself to inspect output.
- Each check is bounded by a 60s timeout.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 6d ago First seen · 54 lines · 54 tokens per session scan A 555e99800631
eval is a skill published in the GitHub repository vasuag09/harness-claude (2 stars, last pushed 1mo ago), licensed MIT. It adds 54 tokens to every session and 764 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 1 finding (runs shell commands). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
ci-tests
Run the test suite for the current repo, auto-detecting Python (pytest/uv), Node (vitest/pnpm), or Rust (cargo test).
git-worktree
Create isolated git worktrees for feature development without switching branches.
git-worktree-status
Check status of background verification tasks running in a git worktree.
typescript-sdk
Implement or modify TypeScript SDK behavior in @composio/core or shared TypeScript packages, including tools, toolkits, sessions, auth configs, connected accounts, modifiers, and generated SDK surfaces. Use for TS runtime/API work; pair with typescript-testing for verification and cross-sdk-parity when Python must…
python-providers
Create, modify, test, or package Python provider adapters under python/providers, including framework-specific dependencies, public imports, type inference, and provider metadata. Use for Python provider work only; use python-sdk for core SDK changes.
python-sdk
Implement or modify Python SDK behavior under python/composio, including tools, toolkits, sessions, auth configs, connected accounts, client integration, and shared Python models. Use for Python core runtime/API work; pair with python-testing and cross-sdk-parity when TypeScript must match.