debugging-and-error-recovery

Debugging and Error Recovery

When to Use

• Tests fail after a code change
• The build breaks
• Runtime behavior doesn't match expectations
• A bug report arrives
• An error appears in logs or console
• Something worked before and stopped working

The Stop-the-Line Rule

When anything unexpected happens:

1. STOP adding features or making changes
2. PRESERVE evidence (error output, logs, repro steps)
3. DIAGNOSE using the triage checklist
4. FIX the root cause
5. GUARD against recurrence
6. RESUME only after verification passes

Don't push past a failing test or broken build to work on the next feature. Errors compound. A bug in Step 3 that goes unfixed makes Steps 4-10 wrong.

The Triage Checklist

Work through these steps in order. Do not skip steps.

Step 1: Reproduce

Make the failure happen reliably. If you can't reproduce it, you can't fix it with confidence.

Can you reproduce the failure?
├── YES → Proceed to Step 2
└── NO
    ├── Gather more context (logs, environment details)
    ├── Try reproducing in a minimal environment
    └── If truly non-reproducible, document conditions and monitor

When a bug is non-reproducible:

Cannot reproduce on demand:
├── Timing-dependent?
│   ├── Add timestamps to logs around the suspected area
│   ├── Try with artificial delays (setTimeout, sleep) to widen race windows
│   └── Run under load or concurrency to increase collision probability
├── Environment-dependent?
│   ├── Compare Node/browser versions, OS, environment variables
│   ├── Check for differences in data (empty vs populated database)
│   └── Try reproducing in CI where the environment is clean
├── State-dependent?
│   ├── Check for leaked state between tests or requests
│   ├── Look for global variables, singletons, or shared caches
│   └── Run the failing scenario in isolation vs after other operations
└── Truly random?
    ├── Add defensive logging at the suspected location
    ├── Set up an alert for the specific error signature
    └── Document the conditions observed and revisit when it recurs