Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add morluto/jacobian --skill verifier-evaluationsgit clone --depth 1 https://github.com/morluto/jacobianWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/morluto/jacobian/verifier-evaluations)<a href="https://agentmods.dev/skills/morluto/jacobian/verifier-evaluations"><img src="https://agentmods.dev/badge/skills/morluto/jacobian/verifier-evaluations.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00023 | $0.00476 |
| Opus 5 | $0.00012 | $0.00238 |
| Sonnet 5 | $0.00005 | $0.00095 |
| Haiku 4.5 | $0.00002 | $0.00048 |
Grade A, and why
verifier-evaluations scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 54 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Verifier Evaluations
A verifier decides a benchmark's mathematical predicate from frozen input and
a bounded submission. It does not grade prose, confidence, tool use, or equality
with one preferred solution. Use harbor-benchmarks only when the task also
needs Harbor packaging, environment changes, or execution guidance.
Establish the contract
Choose the smallest checkable submission: a typed result, a result with a necessary finite witness, or a supported formal proof. Accept mathematically equivalent representations unless canonicalization is an explicit task outcome. The visible instruction and schema must describe every enforced condition and must not leak the solution or derived conclusions.
For a new or changed submission shape, read submission contracts, including witness selection, schema reductions, and independent result fields. Keep instruction, schema, verifier, gold, public contract, and host tests consistent when changing that shape.
Replay and score
Bound input before parsing; enforce exact types and shapes before computation. Replay mathematics against the frozen verifier copy. Malformed submissions must produce a deterministic false predicate and reward artifact, not a host exception. Input binding, mathematical correctness, and declared witness validity remain separate diagnostics where independently observable; required gates combine in reward. Default reward is binary. Partial credit requires explicitly declared, independent mathematical subclaims.
For implementation, diagnostic binding exceptions, artifact/path handling, or regression tests, read replay and attacks. Preserve alternate valid witnesses and a discriminating wrong mathematical claim alongside malformed-input cases. Natural-language proofs need human review unless formalization or an executable certificate makes them checkable.
Complete the repair
Run focused behavioral attacks, the selected Oracle, and the repository's planned Harbor gate when the verifier or task contract changes. Refresh selected Dockerfile checksums after verifier edits through the task preparation workflow. Shared support changes require the affected task-local copies and Oracles; historical snapshots remain unchanged.
What ships with it
3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- yesterday Changed · -93 lines · -45 tokens per session f6150aaf5fc9
- 7d ago First seen · 147 lines · 68 tokens per session scan A aa224a323489
verifier-evaluations is a skill published in the GitHub repository morluto/jacobian (158 stars, last pushed today), licensed MIT. It adds 23 tokens to every session and 476 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
discriminating-experiment-design
Recognize when a benchmark cannot discriminate between methods, construct experiment families that could, sweep structural knobs without sweeping the decision itself, and report a tie that survives as the headline result. Use when all methods score the same on your benchmark, when designing a new evaluation world or…
experiment-audit
A review step that checks whether an experiment's reported results are supported by real data and a sufficiently broad test.
paper-search
Search, download, and read academic papers from 20+ sources (arXiv, PubMed, Semantic Scholar, CrossRef, etc). Use when the user asks to find papers, search for research, look up academic literature, download a paper PDF, or extract text from a paper.
witness
Independently witness that an Allium loop's convergence claim is true and was reached honestly. Use when the user wants to verify a loop's self-report, confirm tests really pass and no generated test was weakened, produce a convergence certificate or witness record, gate CI on a trustworthy signal, or check that an…
circular-validation-audit
Strategy: Run BEFORE building any validator (sandbox/simulation/benchmark). Builds a non-circularity matrix of theory-claim × validator-assumption to detect when a validator would 'confirm' a theory only because it was built on the theory's own premises. A circular validator's PASS carries zero evidential weight.…
paperlab_notebook_regression_baselines
Define notebook baseline, tolerance, and stale-figure rules for PaperLab computational books. Use when notebooks generate figures, tables, or numeric claims that must remain stable as NeqSim and dependencies evolve.