statistical-power

Statistical Power & Sample Size

Overview

Power analysis answers one of the most consequential questions in study planning: how large a sample do you need to reliably detect an effect of a given size, and what could you detect with the sample you can afford? An underpowered study wastes resources and produces inconclusive or irreproducible results; an overpowered one wastes participants, money, and (in clinical work) exposes more people to risk than necessary. Getting this right before data collection is the single highest-leverage statistical decision in a project.

Four quantities are locked together for any given test: sample size (n), effect size, significance level (α), and power (1 − β). Fix any three and the fourth is determined. Every calculation in this skill is some rearrangement of that relationship.

This skill covers the two ways to do power analysis:

• Closed-form formulas (fast, exact for standard tests) — see references/closed_form_recipes.md.
• Simulation / Monte Carlo (works for any design or model you can simulate and analyze) — see references/simulation_based_power.md.

For choosing and converting effect sizes — usually the hardest part — see references/effect_sizes.md.

When to Use This Skill

• Determining required sample size before collecting data (a priori power analysis)
• Finding the minimum detectable effect (MDE) for a fixed, already-determined sample size
• Producing power curves (power vs. n, or power vs. effect size) for a grant or protocol
• Justifying a sample size for an IRB submission, grant, or pre-registration
• Powering designs with unequal group sizes or non-1:1 allocation
• Powering anything without a textbook formula (mixed models, logistic/Poisson regression, cluster-randomized trials, survival analysis, mediation, interactions) via simulation
• Accounting for multiple comparisons, attrition/dropout, or clustering in the sample-size estimate

Installation

Use uv. Pin versions in production; unpinned is fine for exploration.