Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add OutlineDriven/outline-driven-development --skill evaluation-leakage-auditgit clone --depth 1 https://github.com/OutlineDriven/outline-driven-developmentWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/outlinedriven/outline-driven-development/evaluation-leakage-audit)<a href="https://agentmods.dev/skills/outlinedriven/outline-driven-development/evaluation-leakage-audit"><img src="https://agentmods.dev/badge/skills/outlinedriven/outline-driven-development/evaluation-leakage-audit/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/outlinedriven/outline-driven-development/evaluation-leakage-audit"><img src="https://agentmods.dev/badge/skills/outlinedriven/outline-driven-development/evaluation-leakage-audit.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00050 | $0.01205 |
| Opus 5 | $0.00025 | $0.00602 |
| Sonnet 5 | $0.00010 | $0.00241 |
| Haiku 4.5 | $0.00005 | $0.00120 |
Grade A, and why
evaluation-leakage-audit scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 46 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Evaluation leakage audit
Contract
| Field | Bound contract |
|---|---|
| Trigger | Building or reviewing an evaluation, benchmark, or scoring harness, or an explicit request to check it for leakage or contamination. |
| Authority | Read-only: inspect supplied evidence and report in chat; do not mutate files, version control, credentials, paid resources, publications, deployments, or remote state. |
| Side effect | Return only a leakage audit naming where independent ground truth enters and where it does not, with an independence fix for each detected pattern. |
| Done | Every applicable pattern below has been tested; only detected patterns are reported; each finding has evidence and an independence fix; if none fire, the report says no leak was found. |
Inputs
Supply the evaluation or benchmark design and the available artifacts that establish the roles of the model or subject, dataset, labels, scorer, controls, designer, train split, and holdout split. Results, prompts, scoring code, labeler provenance, and subject instructions are optional unless needed to test a pattern. Treat claims about hidden data, independent labels, or private procedures as unverified unless the supplied evidence establishes them.
Procedure
- Bound the audit to the supplied evaluation and evidence. Name the model or subject, scorer, designer, dataset, labels, controls, train split, holdout split, and the origin of any claimed ground truth. Mark unavailable components as unknown rather than inferring them. Done when: every role is named or marked unknown.
- Trace each result backward through scoring and labels to determine whether ground truth enters from a source independent of the system, subject, designer, and outputs being judged. Record both the verified independent entry points and the places where no independent ground truth enters. Done when: every result is traced to its ground truth source with entry points and gaps recorded.
- Test every pattern that the supplied design and evidence make applicable:
- Recall, not reason: determine whether success can come from reproducing memorized benchmark answers rather than deriving an answer. A firing finding must propose fresh or access-controlled items whose answers are independently produced after model training.
- Wrong null hypothesis: determine whether a control removes the label while retaining a proxy or signal that predicts it. A firing finding must propose a control that removes or balances the retained signal while preserving unrelated task structure.
- Shared hallucination: determine whether one generative component validates another without an independent reference. A firing finding must propose externally sourced labels, measurements, or adjudication independent of both components.
- Tautology: determine whether the scorer grades categories, buckets, or criteria that it created from the same outputs. A firing finding must propose criteria fixed before observing outputs and labels produced independently of the scorer.
- Verifier equals designer: determine whether the holdout or verification depends on a private, unreproducible recipe controlled by the experiment designer. A firing finding must propose a preregistered, reproducible procedure or independent verifier with access to auditable evidence.
- Shared-pool bias: determine whether training and holdout labels come from the same labeler pool, allowing shared systematic bias to appear as generalization. A firing finding must propose an independently recruited or independently adjudicated holdout label source.
- Frame injection: determine whether the prompt or question supplies the hypothesis, expected relation, or answer frame being measured. A firing finding must propose neutral wording and blinded alternatives that do not reveal the target hypothesis.
- Demand characteristics: determine whether subjects know the behavior or outcome being measured and can adapt to it. A firing finding must propose blinding, masking, or an unobtrusive measure that withholds the tested expectation without compromising consent. Done when: every applicable pattern is tested with a firing/non-firing determination.
- Report a pattern only when evidence shows that it fires. For each finding, identify the component and evidence, explain how independence is broken, and state the concrete independence fix. Do not rewrite the experiment or widen the audit beyond the supplied scope. Done when: every firing pattern is reported with component, evidence, broken-independence explanation, and fix.
- If no pattern fires after all applicable tests, return a pass that says no leak was found and separately lists unknowns that prevented any pattern from being tested; do not convert missing evidence into either a finding or proof of independence. Done when: a terminal classification of leak found, no leak found, or blocked is returned.
What ships with it
1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 5d ago First seen · 46 lines · 50 tokens per session scan A d13322a66d6f
evaluation-leakage-audit is a skill published in the GitHub repository OutlineDriven/outline-driven-development (52 stars, last pushed 3d ago), licensed Apache-2.0. It adds 50 tokens to every session and 1,205 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
audit-project
Run an iterative multi-agent code audit until critical and high findings are resolved. Use when the user says "audit my code", "find all the bugs", "deep code audit", "iterative review", or "review until clean".
duet
Use when the user invokes /duet, says "pair with me", or faces aesthetic, architectural, or irreversible decisions.
goal-prompt-drafting
Use when asked to draft copy-ready /goal objectives for long-running agents; returns one normalized one-line objective with measurable end state, grounded proof, easy-out invariants, a stop clause, and a Missing list. Not for source or remote-system changes.
handoff-prompt
Use when the user asks for a handoff, delegation, or clipboard-ready prompt for another agent: a standalone path-free prompt copied to the clipboard, confirmed by title. Not for session-snapshot briefs — use handoff; never remote, credential, publish, deploy, or irreversible.
publish-branch
Use when asked to publish the checked-out branch: commit and push it on whatever branch it is, the default branch included. Not for creating branches, PRs, force pushes, or pushing any other branch; when the request excludes the default branch, use commit-push-current.
snippet-image-rendering
Use when the user explicitly names snipgrapher and wants code rendered to a polished PNG, SVG, or WebP at an explicit local path. Probes the installed CLI for supported flags and writes only the named image file. Not for other renderers, publishing, or remote actions.