aistats-reproducibility

AISTATS Reproducibility

Use this before submission and again before camera-ready. Reopen the current CFP and OpenReview forms to confirm whether a reproducibility checklist is required.

Evidence map

• Map each theorem, algorithmic claim, simulation claim, and empirical claim to a verifiable location in the paper, appendix, supplement, or artifact package.
• For theory, state assumptions, proof dependencies, convergence conditions, constants, and failure modes clearly enough for statistical readers.
• For experiments, report datasets, splits, preprocessing, evaluation metrics, baselines, hyperparameter ranges, final selected settings, seeds, repeated runs, compute, and runtime.
• For small performance differences, add uncertainty estimates: standard errors, confidence intervals, paired tests, bootstrap intervals, or repeated trials as appropriate.
• Explain missing code/data honestly and describe how a reader could reproduce the analysis in principle.
• Keep the checklist consistent with the manuscript; contradictions between checklist and paper are review-risk multipliers.

Checklist-to-claim audit table

Marking NA on an item the paper actually triggers is a recognizable AISTATS red flag, because reviewers cross-check checklist answers against the PDF and read contradictions as carelessness about the rest of the paper.

Vignette: a rates-plus-simulation paper

Consider a submission proving posterior contraction rates for a Bayesian nonparametric model, validated by MCMC simulation. Its reproducibility spine: prior hyperparameters and their selection rule, chain length, burn-in, convergence diagnostics, replication seeds, and a statement of which contraction-theorem conditions the simulated model satisfies — plus one honest sentence about the condition it does not.