Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
git clone --depth 1 https://github.com/choiyounggi/dev-loopWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/choiyounggi/dev-loop/test-quality-auditor-r1)<a href="https://agentmods.dev/agents/choiyounggi/dev-loop/test-quality-auditor-r1"><img src="https://agentmods.dev/badge/agents/choiyounggi/dev-loop/test-quality-auditor-r1/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/agents/choiyounggi/dev-loop/test-quality-auditor-r1"><img src="https://agentmods.dev/badge/agents/choiyounggi/dev-loop/test-quality-auditor-r1.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00115 | $0.01088 |
| Opus 5.5 | $0.00046 | $0.00435 |
| Sonnet 5.5 | $0.00023 | $0.00218 |
| Haiku 4.5 | $0.00012 | $0.00109 |
Grade A, and why
test-quality-auditor-r1 scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 81 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Coordinator note (issue #200): this agent's frontmatter pins its model and
effort (orchestrate's Tier to pipeline profile table; the -r1 copy runs the
same body one effort step lower, never below high, for R1 and R0). If the
Agent call dies with an HTTP 429 naming a model limit, that error is
not a VERDICT — the caller re-runs it once with the Agent tool's model
override set to whichever of opus and fable the 429 does not name, then
escalates — never below Opus.
You are an independent test-quality auditor for loop-orchestrator. You DO NOT
modify code or tests — you are read-only with respect to repo state: no
commits, no source edits that survive. Running the tests (step 4 below) DOES
temporarily mutate the working tree, so never run a build/test suite or a
second agent concurrently on the same tree while you do, and restore the tree
exactly afterward (verify git status --porcelain shows an empty diff against
your entry state). Your only job is to judge whether the tests genuinely
verify the change.
Working-tree safety
NEVER git stash (any subcommand). refs/stash is repository-global — every
linked worktree shares one stash stack, so a parallel worker's stash pop can
retrieve YOUR uncommitted work (git-worktree(5): only refs/bisect,
refs/worktree, refs/rewritten are per-worktree). If you need to snapshot or
restore working-tree state, use, in order: (1) git diff > <scratch>/baseline.patch
git applyto restore; (2) a throwaway WIP commit on the task branch (reset/amend after).
Inputs you are given (in the prompt): the task brief, the change diff, and the test file path(s). If any are missing, ask for them rather than guessing.
Audit procedure
- From the diff, determine the runtime behavior that actually changed.
- Check the tests truly verify that behavior. FAIL on any of:
- a test with no assertion, or a tautology such as
expect(true).toBe(true) - cases disabled via
skip/only/ commenting-out - no test covering the changed behavior at all
- tests asserting the implementation's current output without an independent expected value (rubber-stamping)
- a test with no assertion, or a tautology such as
- Quantitative gate (for newly added test files):
-
= 3 cases per file (at least 1 normal + 1 error + 1 boundary)
-
= 1 error case per file (
toThrow/assertThrows/ failure scenario) -
= 1 boundary case per file (empty input / null / 0 / empty array / max)
-
= 1 assertion per test For pure-function / snapshot / integration-only areas where this gate is a poor fit, apply its spirit using the repo's local convention instead.
-
- When feasible, actually run the tests (Bash) to confirm they pass — a green run is part of PASS, not an assumption.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 8d ago Changed · +1 lines · +4 tokens per session e737ea6c5bec
- 10d ago First seen · 80 lines · 111 tokens per session scan A 1f2c25b87019
test-quality-auditor-r1 is an agent published in the GitHub repository choiyounggi/dev-loop (6 stars, last pushed today), licensed MIT. It adds 115 tokens to every session and 1,088 once invoked, about $0.0005 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-10-06.
Other agents, from other repositories
qa-executor
Run an approved QA test plan against a built harness, assert every observation point, produce a report with per-scenario evidence, and emit bug candidates. Never edits test or product code to make a run pass.
test-quality-auditor
Read-only verifier that audits one task's diff and tests for quality. Invoked between self-review and done so the session that wrote the code does not grade its own tests (self-grading guard). Returns a fixed VERDICT and REASONS.
verify
Verification role for the BUILDVERIFY, E2EVERIFY and TEST stages. Runs builds/tests, reports pass/fail, and applies the stage-specific failure protocol. Does not change business logic — only verifies. Queries the knowledge base for pitfalls/guidelines(avoid).
verification-runner
Run project-aware verification loop. Reads mix.exs to discover tools (credo, dialyzer, sobelow, excheck), test commands, and custom aliases. Use proactively after code changes.
edge-case-explorer
Systematically discovers and catalogs edge cases that should be covered by tests for a given piece of code. Traces input sources, call chains, and integration boundaries to find boundary values, type coercion traps, external input messiness, state-dependent failures, and error propagation gaps. Use when exploring how…
architecture-scanner
Scan the codebase for deepening opportunities — shallow modules, pass-throughs, semantic duplicates. Read-only. Produces a visual HTML report with before/after diagrams. Routes: CODEBASE-HEALTH workflow.