boxel: Skill for Claude Code

Flaky Test Fix

A flaky test reported by a CI URL is a request for five things, done in order: prior-art search → Linear issue → worktree → root-cause fix → diagnostic logging. Do not paper over the failure with a retry, an increased timeout, or a skip/todo unless the failure genuinely cannot be diagnosed without more data — and if so, lead with the diagnostics and say so explicitly.

Why this protocol

Flaky tests are uncertain by nature. A fix that looks correct in local repro often does not address the real race / ordering / resource issue that triggers it in CI. Two safeguards counter that:

• Prior-art search catches the common case where the same test has been "fixed" before. Without that context you risk re-doing a fix that already proved insufficient, or undoing an intentional accommodation.
• Diagnostic logging bundled with the fix means that if the fix misses the real root cause, the next CI failure produces enough signal to diagnose it — instead of burning another CI cycle staring at the same opaque output.

Procedure

1. Search for prior fix attempts first

Before touching anything, find out whether this test has been worked on before. Search by test name, file path, and failure signature:

gh search prs "<test name or file>"   # searches open + closed by default
gh pr list --state=all --search "<test name or failure signature>"
git log --oneline -- <path/to/test/file>

If you find a prior PR, read its description, diff, and review thread before designing your fix. You may be looking at a regression, a partial fix, or a class of flakiness someone has already characterized. Surface any prior attempts early so the reporter can weigh in if the new fix should look materially different from the last.

2. Create a Linear issue for the flaky test

Unless the reporter already names a ticket, recommend a Linear issue that tracks the flake, and create it once they agree. Search Linear for the test name first: an existing issue for the same flake is updated, not duplicated. Give a new issue these settings, and set the state of an existing issue too: