Getting it into your agent
This one installs as part of its plugin. Adding the marketplace and installing the plugin brings it with everything else the plugin ships.
/plugin marketplace add navapbc/digital-service-orchestra/plugin install dsoWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/navapbc/digital-service-orchestra/red-test-evaluator)<a href="https://agentmods.dev/agents/navapbc/digital-service-orchestra/red-test-evaluator"><img src="https://agentmods.dev/badge/agents/navapbc/digital-service-orchestra/red-test-evaluator/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/agents/navapbc/digital-service-orchestra/red-test-evaluator"><img src="https://agentmods.dev/badge/agents/navapbc/digital-service-orchestra/red-test-evaluator.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00032 | $0.04647 |
| Opus 5.5 | $0.00013 | $0.01859 |
| Sonnet 5.5 | $0.00006 | $0.00929 |
| Haiku 4.5 | $0.00003 | $0.00465 |
Grade A, and why
red-test-evaluator scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Runs shell commandslowCapability
Expected in a hook, worth knowing in a rule or an instructions file.
1. **Check `SUGGESTED_ALTERNATIVE`**: If the writer's rejection payload includes a `SUGGESTED_ALTERNATIVE` that points to a valid restructuring path (e.g., mock `subprocess.run`, patch `time.sleep`, use small fixture dat How it starts
The opening of the file, as written. The whole thing — 411 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Red Test Evaluator
Startup: Session HEAD Sync (worktree isolation fix)
When dispatched with isolation: "worktree", the Agent runtime creates your worktree branched from origin/main — NOT from the orchestrator's session HEAD. If the orchestrator injected SESSION_BRANCH and SESSION_HEAD into your prompt, sync to the session HEAD as your FIRST action before reading any source files. Bug a951-d6f2-0c21-443f tracks this.
if [[ -n "${SESSION_BRANCH:-}" && -n "${SESSION_HEAD:-}" ]]; then
bash "${CLAUDE_PLUGIN_ROOT}/scripts/worktree-session-head-sync.sh" # shim-exempt: internal orchestration script
if [[ $? -ne 0 ]]; then
echo "ERROR: worktree-session-head-sync.sh failed — aborting" >&2
exit 1
fi
elif [[ -n "${SESSION_BRANCH:-}" || -n "${SESSION_HEAD:-}" ]]; then
echo "WARNING: SESSION_BRANCH/SESSION_HEAD partially set — skipping worktree sync" >&2
fi
When both are unset (orchestrator on main, no session in flight), do nothing — your default origin/main worktree is correct.
Section 1: Role and Identity
You are an opus-tier triage agent for RED test writer failures. When the dso:red-test-writer cannot produce a behavioral test and emits TEST_RESULT:rejected, you receive that rejection payload along with an orchestrator context envelope and determine the correct verdict.
Your job is to answer: Is this rejection legitimate, fixable, or out of scope for automated TDD?
You return exactly one verdict per invocation:
- REVISE — the task list or description needs adjustment; a behavioral test is achievable after revision
- REJECT — the task is outside automated TDD scope; no revision would yield a testable assertion
- CONFIRM — the rejection is legitimate; TDD is genuinely infeasible for this task type
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2mo ago First seen · 411 lines · 32 tokens per session scan A 5cc8b867f4b3
red-test-evaluator is an agent published in the GitHub repository navapbc/digital-service-orchestra (6 stars, last pushed yesterday), licensed Apache-2.0. It adds 32 tokens to every session and 4,647 once invoked, about $0.0001 per session on Opus 5.5. A static security scan graded it A with 1 finding (runs shell commands). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
red-test-evaluator compared with agents from other repositories
| Mod | Type | Stars | Last push | Licence | Scan | Tokens | Works with |
|---|---|---|---|---|---|---|---|
| red-test-evaluatornavapbc/digital-service-orchestra | Agent | 6 | yesterday | Apache-2.0 | A | 32 | Claude Code |
| gem-implementergithub/awesome-copilot | Agent | 41k | today | MIT | A | 22 | Claude Code |
| project-implementerathola/claude-night-market | Agent | 350 | yesterday | MIT | A | 40 | Claude Code |
| nw-ux-designernWave-ai/nWave | Agent | 634 | 4d ago | MIT | A | 31 | Claude Code |
| ai-programmerMRCalderon3D/everything-game-dev-code | Agent | 107 | 5d ago | MIT | A | 19 | Claude Code |
| spring-code-reviewerloiane/specs-driven-development-spring-angular | Agent | 66 | 4d ago | MIT | A | 41 | Claude Code |
| executordatit309/supergraph | Agent | 23 | 12d ago | MIT | A | 25 | Claude Code |