Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/dzhng/duet-agent/write-evalnpx skills add dzhng/duet-agent --skill write-evalgit clone --depth 1 https://github.com/dzhng/duet-agentWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00087 | $0.03629 |
| Opus 5 | $0.00044 | $0.01814 |
| Sonnet 5 | $0.00017 | $0.00726 |
| Haiku 4.5 | $0.00009 | $0.00363 |
Grade A, and why
write-eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 119 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Write an Eval
An eval is only trustworthy if you have seen it both fail for the right reason and pass for the right reason. Writing the assertions, watching them go green once, and moving on is how you ship an eval that passes whether or not the feature works. The standard operating procedure is: pick the outermost entry point, design the assertion so it can only hold when the behavior is present, watch it go green, then falsify — break the production code, confirm the eval goes red with a diagnostic that points at the real path, and restore.
This is the flow used to land evals/state-machine-slash-skill-expansion.eval.ts; read it as the reference implementation for a deterministic wiring eval (the feature either injects the right context or it doesn't).
When the behavior under test is a model tendency rather than deterministic wiring — "the planner doesn't over-reach into implementation", "the sub-agent doesn't drift into chat mode", anything a prompt layer nudges but cannot guarantee — the single-run flow is not enough, because one run is a coin flip. Read evals/state-machine-agent-stays-in-state-scope.eval.ts as the reference for that shape, and follow §6 below in addition to §1–5.
1. Drive the outermost entry point
Per AGENTS.md and the review skill (§13): test behavior through the surface a user actually hits, not internal helpers.
- A unit test on the pure function (e.g.
test/skill-context-resolve.test.tsforresolveSlashSkillPrompt) proves the helper is correct. The eval proves the live wiring invokes it. Write the eval at the layer the unit test cannot reach — the realTurnRunner+startTurnflow, the CLI binary in JSONL mode, or a realcomplete()call. - For state-machine behavior, drive a real
TurnRunnerwith amodedefinition andstartTurnfromtest/helpers/turn-runner-protocol.js.evals/state-machine-agent-cwd.eval.tsandevals/state-machine-slash-skill-expansion.eval.tsare the templates. - For CLI behavior, spawn
bun src/cli.tsin JSONL mode and inspect the emitted events the same way a production subscriber would.evals/inline-slash-commands.eval.tsis the template. - Collect tool calls and assistant text off
runner.subscribestepevents:step.type === "tool_call_start"for calls as they begin (the canonicaltool_callstep carries the echoed input plusisError/output),step.type === "text"for text. Sub-agent (state) events carryevent.origin.kind === "state_machine_agent"; parent events have noorigin. Filter onoriginto attribute a tool call to the right agent.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 119 lines · 87 tokens per session scan A 1fc4e71a144c
write-eval is a skill published in the GitHub repository dzhng/duet-agent (42 stars, last pushed 3d ago), licensed Apache-2.0. It adds 87 tokens to every session and 3,629 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
harness-creator
Build, audit, and improve harnesses that make AI coding agents reliable: AGENTS.md/CLAUDE.md instruction files, feature/state tracking, verification gates, scope boundaries, session handoff, memory persistence, context budgets, tool-permission safety, and multi-agent coordination. Use this whenever a coding agent is…
evaluator-write-qa
Internal Auto-Harness evaluator skill for sprint QA and QA report writing. Use only inside the Evaluator subagent during qa mode.
evaluator-review-contract-parallel
Internal Auto-Harness evaluator skill for parallel sprint contract review before implementation. Use only inside the Evaluator subagent during review mode.
evaluator-write-final-parallel
Internal Auto-Harness evaluator skill for parallel final QA report aggregation. Use only inside the Evaluator subagent during evaluatorfinalparallel.
evaluator-write-qa-parallel
Internal Auto-Harness evaluator skill for parallel sprint QA and QA report writing. Use only inside the Evaluator subagent during evaluatorqaparallel.
evaluator-write-retest-parallel
Internal Auto-Harness evaluator skill for parallel sprint retest and retest report writing. Use only inside the Evaluator subagent during evaluatorretestparallel.