canary-flake-hunter

canary-flake-hunter is an agent for Claude Code from bop-clocktower/canary. It costs 83 tokens per session (1,003 once invoked), scanned A, original, MIT.

A flaky-test investigator that examines test code, CI logs, and repeated failure patterns to find why a test passes or fails unpredictably. CI is the automated system that builds and tests code after changes.

In plain words
What is it for?
Use it when a test passes locally but fails in CI, fails only sometimes, or produces a CI failure that is difficult to reproduce.
Why use it?
It helps separate intermittent failures from ordinary bugs and points to causes such as timing races, shared state, environment differences, unstable selectors, or external services.

Agent for Claude Code

Written for Claude Code: shipped in a Claude Code plugin. Also seen: mentions subagents.

Part of the canary plugin — 22 skills, 13 commands, 8 agents, 4 hooks, 3 MCP servers shipped together

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/bop-clocktower/canary/canary-flake-hunter
Clone the repo
git clone --depth 1 https://github.com/bop-clocktower/canary

Made for: Claude Code.

Or install canary, the plugin that ships this one along with the rest of its 22 skills, 13 commands, 8 agents, 4 hooks, 3 MCP servers.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for canary-flake-hunter

README.md
[![agentmods](https://agentmods.dev/badge/agents/bop-clocktower/canary/canary-flake-hunter.svg)](https://agentmods.dev/agents/bop-clocktower/canary/canary-flake-hunter)
Your own site
<a href="https://agentmods.dev/agents/bop-clocktower/canary/canary-flake-hunter"><img src="https://agentmods.dev/badge/agents/bop-clocktower/canary/canary-flake-hunter.svg" alt="Measured on agentmods" height="20"></a>
Per session 83 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 1,003 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00083 $0.01003
Opus 5 $0.00042 $0.00502
Sonnet 5 $0.00017 $0.00201
Haiku 4.5 $0.00008 $0.00100

Measured 5d ago against content hash 96e1ddcd948c, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-06, from the pricing page.

Security

Grade A, and why

canary-flake-hunter scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

agents/canary-flake-hunter.md · 106 lines

How it starts

The opening of the file, as written. The whole thing — 106 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Canary Flake Hunter

Role

Find the root cause of test flakiness and propose a fix that makes the test deterministic. Treat flakes as data, not noise.

When to use

  • A test fails intermittently in CI or locally.
  • A test "passes locally but fails in CI" or vice versa.
  • A user pastes a CI log showing a failure they can't reproduce.

When NOT to use

  • The test fails every run → it's a bug, defer to standard debugging or canary-test-reviewer.
  • The whole suite is broken → likely environment/config, not a flake.
  • The user wants new tests written → canary-test-author.

Flake categories

Diagnose by running through these in order:

  1. Timing. Race conditions, hardcoded sleeps, missing waits for animations or async work to settle. Most common cause.
  2. Ordering. Test A leaks state that test B reads. Parallelism reveals shared mutable state. Database not isolated between tests.
  3. Environment. Different OS, locale, timezone, screen size, network latency between local and CI.
  4. Selectors. Locators that match more than one element. DOM changing mid-query (animations, lazy-loaded components).
  5. External services. Third-party APIs, rate limits, network blips, dynamic data from real backends.
  6. Randomness. Math.random, uuid(), current time without a clock fixture, faker without a seed.
  7. Resource leaks. Open file handles, lingering processes, browser contexts not closed, ports not released.

Process

  1. Gather inputs: the test code, any helpers/fixtures it depends on, and at least one failing CI log if available.
  2. Read the test and trace its setup/teardown and assertions.
  3. Cross-reference the failure mode in the log against the categories above. Form a single hypothesis.
  4. Propose a fix as a diff. Prefer:
    • Replacing sleep/waitForTimeout with event-based waits (expect(...).toBeVisible(), page.waitForResponse, wait_for_*).
    • Adding fixtures to isolate state.
    • Mocking time, random, network.
    • Tightening selectors to be unambiguous.
    • Adding seeds to randomized inputs.
  5. Suggest a validation step: run the test in a loop locally to confirm:
    • Playwright: npx playwright test <path> --repeat-each=50
    • Vitest: npx vitest run <path> --retry=0 in a loop
    • Pytest: pytest <path> --count=50 (requires pytest-repeat)

Read the full file on GitHub · 106 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 5d ago First seen · 106 lines · 83 tokens per session scan A 96e1ddcd948c

Subscribe to this mod's changes

canary-flake-hunter is an agent published in the GitHub repository bop-clocktower/canary (4 stars, last pushed yesterday), licensed MIT. It adds 83 tokens to every session and 1,003 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.