statistical-test-comparison

statistical-test-comparison

Summary

Compare statistical test stringency and multiple-testing correction outcomes (q-value distributions, test type, variance adjustment) between corrected and uncorrected differential methylation analyses to verify that overdispersion correction produces more conservative significance thresholds.

When to use

When you have run differential methylation analysis in methylKit and need to validate whether overdispersion correction (overdispersion='MN') produces appropriately stringent statistical tests. Apply this skill when comparing a corrected run (with overdispersion='MN' and test='Chisq') against a parallel uncorrected baseline (overdispersion=FALSE) to confirm the correction adjusts variance for excess dispersion and makes p-value/q-value thresholds more conservative.

When NOT to use

• Input methylBase object has <3 replicates per group — Fisher's exact test (not logistic regression or F-test) will be used, and overdispersion correction assumptions may not hold.
• Samples have already been filtered to remove low-coverage bases or PCR bias artifacts — comparison may conflate the effects of coverage filtering with overdispersion correction.
• Goal is exploratory rather than hypothesis-testing — a single uncorrected run may suffice; side-by-side comparison adds computational burden without validation value.

Inputs

• methylBase object (unified methylation data across samples and sites)
• methylKit R package with dataSim() simulation or imported bisulfite sequencing data

Outputs

• q-value distribution from overdispersion='MN' corrected run
• q-value distribution from uncorrected (overdispersion=FALSE) baseline run
• comparative summary statistics (mean, median, range of q-values per method)
• count of significantly differential sites at fixed q-value threshold (e.g., q < 0.01) per method

How to apply

Execute calculateDiffMeth() on a unified methylBase object twice: once with overdispersion='MN' and test='Chisq', and once with overdispersion=FALSE (or default uncorrected mode). Extract the q-value distributions from both runs and compare their central tendencies, ranges, and proportions of sites passing a common significance cutoff (e.g., q < 0.01). The overdispersion='MN' mode applies a scaling parameter φ = X²/(N−P) to adjust variance as φ·n_i·π̂_i·(1−π̂_i), and automatically switches to an F-test from Chi-square, which should result in higher (more stringent) q-values on average. Verify that the corrected method produces visibly elevated median and mean q-values and a smaller proportion of sites meeting a fixed q-value threshold compared to the uncorrected baseline.