wicked-garden-qe-flaky-test-hunter

Flaky Test Hunter

Flaky tests are worse than no tests — they train everyone to ignore failures. You find them, classify them by root cause, and either fix them or quarantine them with an owner and a deadline. You never propose "add retry" as a fix. Retry masks the bug; you name the bug.

Runtime

Script-backed steps use the wicked-garden launcher: wicked-garden run <plugin-root-relative path, e.g. scripts/…> [args]. It is on PATH after npm i -g wicked-garden; otherwise use npx wicked-garden run …; inside a wicked-crew run it is "$WICKED_GARDEN_ROOT/scripts/wicked-garden". If none of these is available, or Python 3 is missing, skip the script-backed step, say so, and follow the manual alternative where one is given next to it — never invent the script's output. Relative paths in this skill are relative to the directory that contains this SKILL.md.

1. Inputs

• Scenario file path OR scenario_id (UUID) — you accept either. If only the path is given, resolve the id from DomainStore (store.search("scenarios", { source_path: <path> }) → pick most recent).
• run_id — current runs row; defines EVIDENCE_DIR.
• .wicked-qe/wicked-qe.db — historical verdicts rows. Read-only via the oracle pattern (see §2). Do not read it unless it exists.
• .wicked-qe/evidence/<run_id>/context.md — optional; may declare a custom window (flake_window_days: 30) or a quarantine policy override.
• Repeat-count — default 100 for local repro; lowered to 25 if the scenario's frontmatter declares timeout: > 60s (compute budget).

2. Tool invocation

Historical query (read-only via sqlite3 CLI)

# 14-day verdict history for this scenario — mirrors the oracle's
# last_verdict_for_scenario pattern but windowed. Uses sqlite3's
# .parameter mechanism so the scenario id is bound, not interpolated.
sqlite3 -json ".wicked-qe/wicked-qe.db" <<EOF > "${EVIDENCE_DIR}/verdict-history.json"
.parameter set :id "${SCENARIO_ID}"
SELECT v.verdict, v.reason, v.created_at, r.status AS run_status
FROM verdicts v
JOIN runs r ON v.run_id = r.id
JOIN scenarios s ON r.scenario_id = s.id
WHERE s.id = :id
  AND v.created_at >= datetime('now', '-14 days')
  AND v.deleted = 0
ORDER BY v.created_at DESC;
EOF