Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add zernie/vigiles --skill test-harnessgit clone --depth 1 https://github.com/zernie/vigilesWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/zernie/vigiles/test-harness)<a href="https://agentmods.dev/skills/zernie/vigiles/test-harness"><img src="https://agentmods.dev/badge/skills/zernie/vigiles/test-harness.svg" alt="Measured on agentmods" height="20"></a>- Socket pass
- Snyk pass
- NVIDIA SkillSpector warn
SkillSpector: 6 findings, up to medium
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- medium Prompt Injection · line 20 Large whitespace padding was detected (a block of blank lines or a long run of spaces). This can push injected instructions below or to the right of the visible area so a human reviewer never sees them while the agent still reads them. Manual review of the hidden content is recommended.Fix: Remove the large whitespace padding (blank-line blocks or long space runs) and review any content hidden below or to the right of it. Keep skill files compact and reviewable so no instructions can be
- medium Prompt Injection · line 20 Large whitespace padding was detected (a block of blank lines or a long run of spaces). This can push injected instructions below or to the right of the visible area so a human reviewer never sees them while the agent still reads them. Manual review of the hidden content is recommended.Fix: Remove the large whitespace padding (blank-line blocks or long space runs) and review any content hidden below or to the right of it. Keep skill files compact and reviewable so no instructions can be
- medium Prompt Injection · line 22 Large whitespace padding was detected (a block of blank lines or a long run of spaces). This can push injected instructions below or to the right of the visible area so a human reviewer never sees them while the agent still reads them. Manual review of the hidden content is recommended.Fix: Remove the large whitespace padding (blank-line blocks or long space runs) and review any content hidden below or to the right of it. Keep skill files compact and reviewable so no instructions can be
- medium MCP Rug Pull · line 105 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 106 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 121 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00103 | $0.02611 |
| Opus 5 | $0.00051 | $0.01306 |
| Sonnet 5 | $0.00021 | $0.00522 |
| Haiku 4.5 | $0.00010 | $0.00261 |
Grade A, and why
test-harness scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 176 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Test the Claude Code harness — the hooks, skills, settings, and CLAUDE.md that steer an agent — as the assembled machine it ships as. vigiles gives three tiers, cheapest first; this skill picks the right one, writes the test, and runs it.
The guiding rule: start at the cheapest tier that can answer the question, and climb only when it genuinely can't. Two of the three tiers need no model and no API key, so they run on every commit for free — reach for the paid real-model tier only when the question actually requires a real model.
Step 0 — Pick the tier (the judgment call)
Match what you're testing to the cheapest tier that can answer it:
| What you're testing | Tier | Cost | API |
|---|---|---|---|
| "Does this hook block/allow event X?" — pure hook logic, every event type (incl. Edit/Write, PreCompact, SessionEnd, SubagentStop) | Unit | free, milliseconds, no claude |
runHook |
| "Is the hook actually wired into the assembled plugin and does it fire in a real session?" | Deterministic | free, no API key (real claude + scripted mock) |
runHarnessTest + scriptModel |
"Did the injected context (a SessionStart hook, a /command) actually reach the model?" |
Deterministic | free, no API key | runHarnessTest → trace.modelRequests / assertRequestContains |
| "Does this skill's description trigger when it should (recall) and stay quiet when it shouldn't (precision)?" | Eval | paid (real model) | measureTriggerRate (+ irrelevantPrompts) → assertTriggerRate({ min, maxFalsePositive }) |
| "Can I measure triggering on a cheaper model and trust it as a floor?" | Eval | paid (two runs) | compareContainment(weak, strong) → formatContainment |
| "Is this exact skill's output any good?" — absolute quality, no on/off baseline (the default for testing one skill) | Eval | paid (real model) | measure({ checks: [judged(rubric)] }) → assertRates({ min }) |
| "Does this harness change move what the agent does, relative to off?" — A/B lift, regression, signal vs noise | Eval | paid (real model) | runEval (arms) + assertSignificant |
Most harness questions — block/allow, wired-in, context-landed — never need a model. Only "does the model trigger / behave differently" needs the eval tier.
⚠️ A trigger-rate of 0% on EVERY prompt is a wiring bug until proven otherwise.
It reads like a verdict on the description, and three separate setup mistakes
produce it: a bare id in fired where the namespaced <plugin>:<skill> is
required; pluginDir where a loose .claude/skills needs skillsDir; and a
missing fixture, since a run starts in an empty directory and a prompt about
a file that isn't there is one the model is right to decline. Rule all three out
before reporting it. (A partial rate is a real number — don't second-guess it.)
What ships with it
3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 4d ago Changed · -212 lines 45d33e5a4881
- 8d ago First seen · 388 lines · 103 tokens per session scan A 50b54c4e3ac1
test-harness is a skill published in the GitHub repository zernie/vigiles (15 stars, last pushed today), licensed MIT. It adds 103 tokens to every session and 2,611 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
caliber-testing
Writes Vitest tests following project patterns: tests/ directories, vi.mock() for module mocking with vi.hoisted() for test-time factories, global LLM mock from src/test/setup.ts, environment variable save/restore in beforeEach/afterEach, vi.clearAllMocks() lifecycle, and test file organization. Use when user says…
reproducibility-validate
Run a workflow multiple times and compare outputs to produce a similarity score and pass/fail verdict.
eval-agent
Run evaluation tests against an agent to assess quality and archetype resistance.
eval-workflow
Run evaluation tests against a multi-agent workflow to assess orchestration quality and failure archetype resistance.
auto-test-execution
Automatically execute tests when code-generating agents modify source files, enforcing the execute-before-return pattern.
implement
Implement a technical design by decomposing it into dependency-ordered vertical slices, executing each with TDD red-green, reviewing each via an isolated sub-agent, and persisting progress to a state file so work survives session restarts. Use when the user has a tech design (doc or settled conversation) and says…