hypothesis-first

Hypothesis-First (Scientific TDD)

Overview

Register the hypothesis first. Run the analysis. Interpret against the prediction.

Core principle: If you didn't commit your hypothesis before seeing the data, you don't know if you're doing science or storytelling.

Violating the letter of the rules is violating the spirit of the rules.

When to Use

Always:

• Any statistical analysis
• Any model comparison or validation
• Any experiment, pilot, or feasibility study
• Any reanalysis of existing data
• Any "quick look" at results

Exceptions (discuss with your PI or collaborator):

• Purely exploratory analyses — permitted, but MUST be labeled as exploratory in all outputs and the manuscript
• Computational benchmarking with no inferential claim

Thinking "skip registration just this once"? Stop. That's rationalization.

The Iron Law

NO DATA ANALYSIS WITHOUT A REGISTERED HYPOTHESIS FIRST

Ran the analysis before registering? Delete the results. Start over.

No exceptions:

• Don't keep results as "reference"
• Don't "adjust" your hypothesis to match what you found
• Don't peek at effect sizes to calibrate your prediction
• Don't label post-hoc HARKing as "hypothesis-driven"

Register first, then run. Period.

REGISTER-EXECUTE-INTERPRET

digraph hypothesis_first_cycle {
    rankdir=LR;
    register [label="REGISTER\nState H1, H0, outcome,\ntest, threshold.", shape=box, style=filled, fillcolor="#ffcccc"];
    verify_register [label="Inline verify\nregistration\nis complete", shape=diamond];
    subagent_review [label="Dispatch\nregistration-reviewer\n(fresh subagent)", shape=box, style=filled, fillcolor="#ffaaaa"];
    commit [label="git commit\n(only if Approved)", shape=box, style=filled, fillcolor="#ff8888"];
    execute [label="EXECUTE\nRun pre-specified\nanalysis exactly.\nNo deviations.", shape=box, style=filled, fillcolor="#ccffcc"];
    verify_execute [label="Any deviation\nfrom plan?", shape=diamond];
    document_deviation [label="Document as\nEXPLORATORY\nlabel deviation", shape=box, style=filled, fillcolor="#ffffcc"];
    interpret [label="INTERPRET\nCompare to prediction.\nReport ALL results.\nNo selective reporting.", shape=box, style=filled, fillcolor="#ccccff"];
    next [label="Next\nhypothesis", shape=ellipse];

    register -> verify_register;
    verify_register -> subagent_review [label="complete"];
    verify_register -> register [label="incomplete:\nfinish it"];
    subagent_review -> register [label="Issues Found:\nfix and re-review"];
    subagent_review -> commit [label="Approved"];
    commit -> execute;
    execute -> verify_execute;
    verify_execute -> document_deviation [label="yes"];
    verify_execute -> interpret [label="no"];
    document_deviation -> interpret;
    interpret -> next;
    next -> register;
}