Canary — AI-powered test automation agent. Generate, review, recommend, and de-flake tests across Playwright, Vitest, Pytest, and k6, as a Claude Code plugin.
Skill Claude CodeCodex
Add a new testing framework to Canary's registry end-to-end — aligns the classifier↔registry contract, authors the ts/src/data/frameworks/registry.json entry, validates the execution command, and ensures every classifier testtype still resolves to a non-null framework. Use for "add support for a new framework", "add…
Skill Claude CodeCodex
Temporal-dependency linter for test files — statically flags tests that lean on wall-clock time, a real delay, or the local timezone, the ones that pass all day and fail at midnight, across a DST boundary, or on Feb 29. Suppresses itself when the file already installs a frozen clock (fake timers, freezegun…
Skill Claude CodeCodex
Vacuous-test detection — finds tests that PASS WITHOUT PROVING ANYTHING: an assertion that compares a value with itself, a test that never invokes the target it claims to cover, and a test whose every assertion is an absence observed on a bystander rather than on the code under test. Use when the user says "why did…
Skill Claude CodeCodex
Analyses a test suite for CI readiness: coverage depth, flakiness, assertion quality, critical path coverage, and suite runtime. Accepts documented failures (quarantined tests with linked open issues count as verified). Investigates config/auth failures using the consuming repo's declared usercatalogskill.
Skill Claude CodeCodex
Scaffold .canary/company.json — the org-specific pointer file (Confluence spaces, Jira projects, internal docs/domains, MCP servers, dashboard, and the user-catalog skill) that canary-ci-ready and canary-failure-impact silently assume already exists. Use when the user says "set up company knowledge", "bootstrap…
Skill Claude CodeCodex
Risk-based test prioritisation. Given a codebase or diff, identifies which areas carry the most risk using git churn, downstream dependents, business-critical signals, and existing coverage depth. Produces a ranked list with recommended test types per area.
Skill Claude CodeCodex
Given a feature description, function signature, or existing test suite, surfaces edge cases worth testing across six categories. Explanation depth scales to user skill level. Optionally focuses on critical areas when critical-areas.json is present.
Skill Claude CodeCodex
Surface test failures fast and loud — audit a Playwright config for fail-fast knobs (maxFailures/forbidOnly/retries) and print a loud, categorized failure digest to the CI log + ::error annotations at run end, failing the step so the signal can't be missed. Self-contained (bundles its own Playwright JSON parser and…
Skill Claude CodeCodex
For a given test, function, or code path, traces downstream effects and produces a severity label. Investigates config/auth failures using the consuming repo's declared usercatalogskill. Optionally focuses on critical paths when critical-areas.json is present.
Skill Claude CodeCodex
Fleet-wide test health summary across suites — flaky tests, failure spikes, cross-suite common failures, and regression candidates from the run-history store. Use when the user asks "how healthy is our test fleet", "fleet-wide flake report", "any regressions this week", "failure spikes across suites", or "canary…
Skill Claude CodeCodex
Generate a framework-appropriate test from a natural-language requirement by routing through Canary's classify → recommend → generate pipeline (the /canary-write-test slash command), writing the test under tests/generated/ and optionally executing it. Use for "write a test for X", "I need an API test that does Y"…
Skill Claude CodeCodex
Instrument a Playwright run with OpenTelemetry and emit a run.json artifact correlating every test to the outbound HTTP requests it made — "which test made which request?" — with zero manual bookkeeping in test code. Trace-only v1 contract, additive-safe for future pytest/k6/node producers. Self-contained (bundles its…
Skill Claude CodeCodex
Quarantines deleted and newly-skipped tests instead of letting them vanish. Captures every removed or skipped test with provenance (who, when, which commit, why) into an append-only ledger, and alarms in exactly one case — the deletion removed the last coverage of a symbol critical-areas.json marks high-risk. Silent…
Skill Claude CodeCodex
PR/pre-commit test-guardian orchestrator: runs the deterministic Tier-0 diff-coverage pass, audits affected tests via canary-test-reviewer (Tier 1), and — at the desk with authorTests opt-in — authors missing tests via canary-test-author (Tier 2), staging them and blocking the commit once for human review. Use to…
Skill Claude CodeCodex
Move a generated test from tests/generated/ into the committed test suite — reviews it for correctness, drops the generation header, relocates it to the matching suite directory, and confirms it runs in the project's normal test flow. Use for "promote this test", "commit this generated test", "move this test into the…
Skill Claude CodeCodex
Order-dependence and isolation detector for test suites. A Tier-1 static scanner flags the shared-state smells which predict order-dependent tests - a module-level mutable a test writes to, a setup with no matching teardown, a mutated process singleton, an order-coupled name - with no test execution, so it runs…
Skill Claude CodeCodex
Configure the Harness Engineering guardrails in a new Canary project (or a fork) — installs the harness CLI, initialises harness.config.json, wires up the five required CI workflows, generates the initial security ledger, and verifies all gates pass on a clean repository. Use for "set up harness for this project"…
Skill Claude CodeCodex
Differential ("shadow") parity testing — run the SAME invocation through a baseline and a candidate, normalize away irrelevant noise (ANSI, timestamps, temp paths, version banners, SHAs), diff, and flag divergences. Use to prove a behavior-preserving change actually preserves behavior: a language/framework port…
Skill Claude CodeCodex
Run the canary ship gate on a finished change: spawn parallel adversarial reviewers on the diff, resolve every confirmed finding with a regression test, then commit, open a PR, and squash-merge while watching CI to green. Use this whenever an implemented, locally-green change on a feature branch should be merged …
Skill Claude CodeCodex
Multi-phase orchestrator that composes canary-ci-ready, canary-critical-areas, canary-edge-case-discovery, canary-failure-impact, and canary-write-test into a sequential pipeline with a convergence loop and health report.
Skill Claude CodeCodex
Playwright JSON results → Markdown + JSON test report. Summarises passed, failed, flaky, and skipped counts with a per-failure error block and a summary table. Exits non-zero when any test failed so the CI step fails on real failures. Complements canary-fail-fast (which aborts early); this skill summarises the full…