debug-test-failure

Diagnosing a failing test

A failing test is a measurement, not an obstacle. Work the failure as evidence before touching any code. The goal is to know why it fails before you change anything — a fix applied without a diagnosis usually moves the failure rather than removing it.

1. Read the failure, all of it

Start from the raw output, not your memory of what the test does.

• Find the assertion that failed and the exact line it failed on. Frameworks bury this under stack frames; the first frame inside your code is usually the one that matters.
• Record the expected and actual values verbatim. The shape of the mismatch tells you a lot: off-by-one, wrong type, empty vs populated, None/null where a value was due, a timestamp or ordering difference.
• Note whether it is an assertion failure, an uncaught exception, a timeout, or a panic/crash. These have different root-cause families — a timeout is rarely the same kind of bug as a wrong return value.
• Scan for a second failure. The first error often causes the rest; fix the first and the cascade may clear.

2. Confirm it is the test you think it is

Run the single test in isolation before forming any theory:

# pick the form your runner uses
cargo test path::to::test_name -- --exact --nocapture
pytest tests/test_mod.py::test_name -x -vv
go test ./pkg -run '^TestName$' -v

Two outcomes, two meanings:

• Fails in isolation — the bug is in the unit under test or this test's own setup. Good; you have a clean reproduction.
• Passes in isolation, fails in the suite — this is a test-interaction bug: shared global state, leaked fixtures, ordering dependence, a database or temp file not reset between tests, or a parallel-execution race. Hunt the state that leaks across tests, not the assertion.

3. Isolate the assertion

Narrow the failure to the smallest claim that is false.

• If the assertion compares a large structure, compare fields one at a time until you find the field that differs. Assert on that field alone.
• If it loops, find the first iteration that fails and pin the inputs for that iteration.
• Replace a complex expected value with an inline literal you computed by hand. If your hand-computed value also fails, the bug is in the code; if it passes, the bug was in how the test built its expectation.