Borrowing it
Nothing to install: this file belongs to lebek/modmixer. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/lebek/modmixer/main/.claude/skills/harness-verify/SKILL.mdgit clone --depth 1 https://github.com/lebek/modmixerWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/lebek/modmixer/harness-verify)<a href="https://agentmods.dev/skills/lebek/modmixer/harness-verify"><img src="https://agentmods.dev/badge/skills/lebek/modmixer/harness-verify.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00104 | $0.01803 |
| Opus 5 | $0.00052 | $0.00901 |
| Sonnet 5 | $0.00021 | $0.00361 |
| Haiku 4.5 | $0.00010 | $0.00180 |
Grade A, and why
harness-verify scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 6d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 108 lines — stays where its author put it; the contents beside it link to each section on GitHub.
harness-verify — headless A/B turn replay
Replays a recorded ModMixer chat up to a chosen point, then runs the agent's
next turn headlessly against a chosen model — once per variant, repeated
N times — and reports what the agent did (asked the user vs. fired a
side-effecting tool like run_test_cycle). This is how you get runtime
evidence that a prompt/tool/hint change actually moves model behavior,
especially for non-frontier models (Kimi etc.) where "it's in the system
prompt" is not the same as "the model obeys it".
It runs the real system prompt, the real tool set, and the real model + credentials — but in an isolated session with side-effecting tools swapped for recording stubs, so nothing edits files, builds, launches RimWorld, or touches the user's conversation store.
When to use
- You changed how the agent should behave (a system-prompt rule, a tool description, a tool-result hint, a model) and want to see the effect.
- You want an A/B: same history + model, with vs. without your change.
- The surface is the agent's decision, and driving the GUI by hand is slow, costs RimWorld launches, and isn't repeatable.
Not for: pure mechanical changes with no behavioral question (a typecheck is enough), or anything where you'd need many full agentic turns with real file mutation (this stubs the side-effecting tools).
Run it
# Always dry-run first — proves the pipeline (Electron boot, model
# resolution, session reconstruction, prompt build) with ZERO model cost.
node C:\Users\peter\projects\modmixer\scripts\harness\run.mjs `
--title thunderstorm --until "make it sunny" --dry
# Live A/B: baseline (no change) vs fix (change), 3 turns each.
node C:\Users\peter\projects\modmixer\scripts\harness\run.mjs `
--title thunderstorm --until "make it sunny" `
--model moonshotai/kimi-k2.6 --provider openrouter --repeat 3
Use an absolute path to run.mjs (the runner finds the repo from its own
location; the shell cwd is irrelevant and has drifted before).
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 6d ago First seen · 108 lines · 104 tokens per session scan A 99f2d716018a
harness-verify is a skill published in the GitHub repository lebek/modmixer (20 stars, last pushed 24d ago), licensed MIT. It adds 104 tokens to every session and 1,803 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
writing-tests
How to write a Kiro Crew backend test that has NO side effects and does not flake. Use when adding, editing, reviewing, or debugging a pytest test in the Kiro Crew source repo: which conftest is under your file, what leaks (temp dirs, the real data home, /.kiro, cron, threads, child processes), how to tell which of…
pod-e2e
ONLY for developing Kiro Crew itself -- if the project you are working on is anything else, ignore this skill: it drives Kiro Crew's own pod tooling, which does not exist in another repository. Runs end-to-end tests (backend API + frontend Playwright) for a Kiro Crew feature worktree against an ISOLATED throwaway pod…
ai-discover
Parallel discovery of performance hotspots (perf track) and failure surfaces (bug track) for the auto-improvement loop. Fans out one subagent per hot-path area or failure surface; each returns ONE concrete, behavior-preserving fix candidate (perf) or a reproducing test plus fix (bug). Discovery only — no code changes…
relay-80-100-workflow
Use when writing agent-relay workflows that must fully validate features end-to-end before merging. Covers the 80-to-100 pattern - going beyond "code compiles" to "feature works, tested E2E locally." Includes repair-before-failure validation gates, mandatory sequential Claude-then-Codex fresh-eyes review/fix loops…
adk-verify-snippets
Checks that every Python code block in a Markdown file actually compiles and runs, by extracting each block to a temporary file, executing it in an isolated subprocess, and writing a pass/fail report with per-snippet coverage. Use when the user asks to verify, test, or validate the code samples in a README, a guide…
adk-setup
Sets up a local ADK Python development environment in a git clone of the open-source adk-python repository: a uv virtual environment, all dependency extras, pre-commit hooks, and a first unit-test run. Runs only when explicitly requested, never on its own. Use when asked to set up, bootstrap, or repair a development…