thinking-scientific-method

Scientific Method (Hypothesis Differential)

When a symptom could come from several places, enumerate competing falsifiable hypotheses and spend the cheapest observation on the one that best discriminates among them. After each observation, keep only hypotheses that still fit, then prefer the survivor with the fewest unsupported assumptions as the working explanation.

When to Use

• A bug, incident, or anomaly has more than one plausible cause.
• You can observe code, logs, diffs, traces, tests, configs, or data now.
• You must localize the faulty file, function, branch, config, or invariant before fixing.
• Competing explanations fit the same surface facts and you need a discriminating check.

When NOT to Use

• Cause is already obvious from a single stack, failing test, or recent diff — fix directly.
• Only one plausible hypothesis exists — test it; do not invent rivals for ritual.
• No observation is possible yet — obtain access first; do not speculate a localization.
• Multi-week experiments, product A/B tests, or policy trials — this skill is for agent-now checks.
• Fault is already localized and you need systemic root/prevention depth — use five-whys-plus.
• Selective "only these objects/times" defects better suited to IS/IS-NOT comparison — use Kepner-Tregoe.
• Representation (doc/dashboard) may be stale versus reality — verify territory with map-territory first, then resume hypotheses.

Procedure

1. State the symptom precisely. Capture failing behavior, scope, timing, environment, and constraints. Separate observation from interpretation.
2. Enumerate 2–5 competing hypotheses. Name specific files, functions, configs, input conditions, or invariants. Reject vague buckets ("backend issue"). If no serious alternative remains after a deliberate check, exit this differential and test or fix the sole evidenced cause directly; never fabricate a rival to continue the procedure.
3. Name falsifiers and cheap observations before looking. For each hypothesis: what result drops it, and what read/grep/diff/log/test check can you run now. Prefer observations available immediately over deploys, canaries, or long waits.
4. Rank observations by discrimination × cheapness. Run the cheapest check that best separates the top contenders. Do not deep-dive the favorite first if a cheap cross-check would kill alternatives.
5. Update after each observation. Drop falsified hypotheses. Among survivors that still fit all evidence, prefer the one with the fewest independent unsupported assumptions (extra components, rare timing, external dependencies). Parsimony ranks survivors after fit; it never rescues a leaner hypothesis that evidence already contradicts. Escalate complexity only when simpler survivors are ruled out.
6. Localize and stop. When one hypothesis has direct supporting evidence and key alternatives are ruled out, name the file/function/config to change and the evidence that localizes it. Stop analyzing once localization is direct.