agentic-reviewer: Skill for Claude Code

Review Statistics

Evaluate the statistical rigor and data presentation quality of an academic paper.

Prerequisites

Before starting, read:

1. The paper file
2. context/<journal-slug>/scope-and-criteria.md if the journal has specific statistical requirements

Review Checklist

Study Design & Power

• Was an a priori power analysis conducted?
• Is the sample size adequate for the claimed effects?
• Were there multiple comparisons that required correction?

Statistical Methods

• Are the statistical tests appropriate for the data type and research question?
• Are assumptions of the tests checked and reported (normality, homoscedasticity, independence)?
• Are non-parametric alternatives used when assumptions are violated?
• Are model fit indices reported (for regression/SEM models)?
• Is the software/package used for analysis stated?

Effect Sizes & Confidence Intervals

• Are effect sizes reported (not just p-values)?
• Are confidence intervals provided?
• Is practical significance discussed, not just statistical significance?

Multiple Comparisons

• Are multiple comparison corrections applied where needed (Bonferroni, FDR, etc.)?
• Is the family-wise error rate controlled?
• Are pre-registered analyses distinguished from post-hoc analyses?

Data Presentation

• Are results clearly presented in tables and figures?
• Is there redundancy between text, tables, and figures?
• Are standard deviations, not just means, reported?
• Are exact p-values reported rather than just significance thresholds?

Reproducibility

• Is analysis code available or described in sufficient detail?
• Are data availability statements included?
• Could the analysis be replicated from the information provided?

Red Flags

• p-hacking indicators (optional stopping, selective reporting)
• HARKing (Hypothesizing After Results are Known)
• Data dredging or fishing expeditions
• Selective reporting of analyses
• Overinterpretation of marginal results