simulation-based-validation

simulation-based-validation

Summary

Generate synthetic methylation data using dataSim() to benchmark and validate statistical methods such as overdispersion-corrected differential methylation analysis. This skill enables controlled testing of analysis parameters and their effects on stringency and accuracy before applying methods to real data.

When to use

When you need to verify that a statistical correction (e.g., overdispersion adjustment in calculateDiffMeth with overdispersion='MN') produces expected changes in test stringency, or when you want to validate that a new analysis workflow produces correct q-value distributions and variance adjustments under known ground-truth conditions.

When NOT to use

• When you have sufficient real experimental replicates and real biological effect sizes — simulation is no substitute for validation on actual data with true biological variance.
• When your primary goal is differential methylation discovery on real samples — simulation-based validation is a method-vetting step, not a substitute for direct analysis.
• When computational resources are severely constrained and you must prioritize analysis speed over methodological validation.

Inputs

• methylKit dataSim() parameters (number of replicates, number of methylation sites)
• methylBase object (simulated or real methylation data with coverage and methylation percentage columns)

Outputs

• methylDiff object with q-values and test statistics from uncorrected run
• methylDiff object with q-values and test statistics from corrected run (overdispersion='MN')
• q-value distribution comparison (mean, median, and percentile values)

How to apply

Use methylKit's dataSim() function to generate a synthetic methylBase object with controlled parameters (e.g., 6 replicates and 1000 methylation sites). Run calculateDiffMeth() on this object twice: once with your method of interest (e.g., overdispersion='MN' with test='Chisq') and once with a baseline uncorrected configuration (overdispersion=FALSE or default). Extract and compare the resulting q-value distributions between runs to verify that the corrected method produces higher average q-values (more stringent multiple-testing correction) and that the variance adjustment factor φ = X²/(N-P) is correctly applied, confirming the expected shift from Chi-square to F-test behavior.