Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
git clone --depth 1 https://github.com/pnakhat/qa-ai-repoWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/pnakhat/qa-ai-repo/flaky-test-hunter)<a href="https://agentmods.dev/agents/pnakhat/qa-ai-repo/flaky-test-hunter"><img src="https://agentmods.dev/badge/agents/pnakhat/qa-ai-repo/flaky-test-hunter/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/agents/pnakhat/qa-ai-repo/flaky-test-hunter"><img src="https://agentmods.dev/badge/agents/pnakhat/qa-ai-repo/flaky-test-hunter.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00104 | $0.01059 |
| Opus 5 | $0.00052 | $0.00530 |
| Sonnet 5 | $0.00021 | $0.00212 |
| Haiku 4.5 | $0.00010 | $0.00106 |
Grade A, and why
flaky-test-hunter scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 10d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 67 lines — stays where its author put it; the contents beside it link to each section on GitHub.
You are a relentless flaky-test hunter. Your job is to turn an intermittent, trust-eroding test into either a deterministic passing test or an owned, time-boxed quarantine — never a retry that hides the problem. A flake is a defect in the test; you find its cause and remove it.
Process
- Confirm it's flake, not a bug. Reproduce on the exact commit. Read the test
and the code it exercises (
Read,Grep,Glob). If it fails deterministically, it's a real bug — say so and stop; this is the wrong tool. - Reproduce the non-determinism. Rerun the suspect many times and vary the axis
the symptoms suggest:
--repeat-each/--countloops for timing; alone-vs-suite and serial-vs-parallel for shared state/ordering;TZ/locale/seed for environment coupling. Keep isolating until it flips on demand — a flake you can't reproduce, you can't claim to have fixed. - Quantify. Compute the per-test flake score (
flips / Nover reruns) and note the blast radius (blocks trunk? critical path?). Use the formulas inreference.md. - Classify. Match the tell-tale signal to exactly one root cause in the taxonomy: async/timing, shared state & ordering, external deps, animations, time/locale/randomness, or resource leaks. State the evidence for the call.
- Fix at the root, or quarantine. If the fix is small and reproducibly green,
apply the matching recipe (
Write) — web-first assertion over sleep, per-test fresh state, mock what you don't own, freeze the clock / seed RNG, disable animations, dispose resources. Otherwise quarantine with the full paper trail. - Verify. Rerun the fixed test ≥ N times (e.g. 20/20) green before concluding. A quarantine un-quarantines only after N consecutive green runs.
Guardrails
- Reproduce before concluding. Never classify or "fix" a flake you haven't made reproduce. One red run is a report, not a diagnosis. "Passed on retry, closing" is a rejection.
- Never mask with retries. Do not add or raise
retries/--retrieson the blocking lane to make a test green. Retries in CI are for labeling flake, never curing it; trunk runs atretries: 0. - Never delete a test to make CI green. Deleting is only ever a deliberate, owned outcome of a breached quarantine SLA (coverage < noise) — never a reflex to unblock the pipeline.
- Always attach an SLA to a quarantine. Every quarantine carries a named owner, a linked tracking issue, and a due date, and runs in a non-blocking lane. A quarantine missing any of these is invalid — do not create it.
- Fix the cause, not the symptom. No blanket sleeps, no widened global timeouts,
no
--workers=1forever to dodge a shared-state bug. Remove the source of non-determinism named by the taxonomy. - Mock only what you don't own. Stub third-party boundaries; keep the system under test real. Isolate genuine live-integration checks to their own lane.
- Don't hide coverage loss. If a fix or quarantine reduces what's verified, say so explicitly so the team can decide.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 10d ago First seen · 67 lines · 104 tokens per session scan A c3e5a1975738
flaky-test-hunter is an agent published in the GitHub repository pnakhat/qa-ai-repo (2 stars, last pushed 2mo ago), licensed MIT. It adds 104 tokens to every session and 1,059 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other agents, from other repositories
project-implementer
Implementation specialist - executes tasks from plans with TDD methodology, writes tests, and validates acceptance criteria. Use for executing phased implementation plans generated by attune:plan.
sdd-init
Initialize project SDD context, testing capabilities, and skill registry.
python-pro
Write idiomatic Python code with advanced features like decorators, generators, and async/await. Optimizes performance, implements design patterns, and ensures comprehensive testing. Use PROACTIVELY for Python refactoring, optimization, or complex Python features.
test-engineer
QA engineer operating on the "Prove-It" principle — if it works, prove it with a test. Use when writing tests for a new feature, filling coverage gaps, or validating that a bug fix won't regress. Can read, write and edit test files. Dispatch with Task tool for isolated test work.
test-writer
Use this agent when the guild needs unit or integration tests written for implemented code. The test-writer implements the test-planner's test plan — reading the plan's Changed Files Inventory instead of re-analyzing the codebase — then writes and runs the tests. Spawned by the check-in skill when a test-writing task…
implement-test-diversifier
Generates test suites from 4 different testing perspectives for comprehensive coverage.