Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add synaptiai/synapti-marketplace --skill holdout-validationgit clone --depth 1 https://github.com/synaptiai/synapti-marketplaceWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/synaptiai/synapti-marketplace/holdout-validation)<a href="https://agentmods.dev/skills/synaptiai/synapti-marketplace/holdout-validation"><img src="https://agentmods.dev/badge/skills/synaptiai/synapti-marketplace/holdout-validation/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/synaptiai/synapti-marketplace/holdout-validation"><img src="https://agentmods.dev/badge/skills/synaptiai/synapti-marketplace/holdout-validation.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00130 | $0.01125 |
| Opus 5 | $0.00065 | $0.00562 |
| Sonnet 5 | $0.00026 | $0.00225 |
| Haiku 4.5 | $0.00013 | $0.00112 |
Grade A, and why
holdout-validation scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 74 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Holdout Validation
Contract
Iron law: the holdout set stays hidden; scenario IDs only, never names or descriptions. Invoked by /flow:start Phase 4 VERIFY step 4, /flow:address Phase 4 step 3, and /flow:review Phase 3 fan-out (twice in Path A, once in Path B) with self-review findings, an evidence bundle draft, and a file list. Returns Holdout validation: PASS or FINDINGS with P1/P2/P3 rows citing file:line, mapped to visible acceptance criteria, consumed by verdict-judge. Permitted skips: none; with a missing input, evaluate what is available and note the gap, halting only when no file list is given.
Inputs
- Self-review findings: the code-reviewer's P1/P2/P3 findings
- Evidence bundle draft: per-criterion evidence, including
### Test inputs and expected valuesand### Risk map coverage - File list: every file modified or created on the branch
Schema: schemas/holdout-validation/input-schema.json.
Process
- Load scenarios for each criterion type present from
templates/holdout-scenarios/{behavioral,api,error,data}.md. Number them by document order across loaded files (scenario-1 to scenario-N); skip and note a missing type. - Parse claims. Extract each finding's and evidence entry's claim and file references. A claim with no file/line citation is a bare assertion: automatic P2.
- Cross-reference claims. Read each cited location; record CONFIRMED or CONFLICT. "Test added for X" needs assertions that verify X, not a test that names X.
- Cross-reference expected values. For each criterion, read
### Test inputs and expected valuesand open every cited test. ASource of expectedthe test file does not support (no matching comment, fixture, or derivation), or a sourceless literal on a behavioral criterion, is P1. - Cross-reference risk coverage. Read
### Risk map coverage. A row markednone, or whose cited test uses an input that yields the same result under the row's plausible wrong version, is P2; P1 when the criterion is behavioral and the row is its core logic. - Evaluate every scenario against file state and claims, even seemingly inapplicable ones. A FAIL maps to a visible criterion, described without reference to the scenario.
- Emit findings; run the security check.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago Changed · -105 lines · +10 tokens per session 71206b6d95a7
- 9d ago First seen · 179 lines · 120 tokens per session scan A 1c8dac1595e2
holdout-validation is a skill published in the GitHub repository synaptiai/synapti-marketplace (6 stars, last pushed yesterday), licensed Apache-2.0. It adds 130 tokens to every session and 1,125 once invoked, about $0.0006 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
quality-checklist
Validate implementation quality through custom checklists, scoring against constitution standards, specification coverage, and producing remediation recommendations.
brooks-test
Test quality review drawing on twelve classic engineering books — with primary focus on xUnit Test Patterns, The Art of Unit Testing, How Google Tests Software, and Working Effectively with Legacy Code — that diagnoses structural problems in an existing test suite: brittleness, mock abuse, coverage illusions, slow…
check
Confirm a change before merge. /check verify drives the real app to prove behavior against the spec (every acceptance criterion met, every surface built). /check review runs a senior code review on a fresh model, one that did not write the code. Verify after /develop, review before a PR. Writes to docs/reviews/, never…
testing-laravel
Writes Laravel tests using PHPUnit. Use when "write tests", "add tests", "phpunit", "laravel test", "feature test", "unit test", "mock", "factory", or testing controllers, models, services, actions, jobs, artisan commands, or API endpoints.
testing-react
Writes React/TypeScript tests using Vitest and React Testing Library. Use when "write react tests", "vitest", "component test", "hook test", "RTL", "testing library", "snapshot test", or testing React components, hooks, and utilities.
x-qa-gate
A quality-review gate for completed development tasks. It checks the code against the request, design, tests, verification evidence, and current changes, using different review rounds for different risk levels.