duplicate-read-filtering-and-normalization

duplicate-read-filtering-and-normalization

Summary

Remove redundant reads from ChIP-Seq data and normalize read counts across samples to enable unbiased statistical comparison. This skill is essential before peak calling, as duplicate reads inflate coverage at single genomic loci and violate the assumption of independent sampling required for p-value and q-value calculations.

When to use

Apply this skill when you have raw ChIP-Seq and control BED files with potential PCR duplicates or unequal sequencing depths. Duplicate filtering is mandatory before estimating fragment length (predictd) or generating coverage pileups. Normalization is required whenever ChIP and control samples have different total read counts, so that downstream statistical tests (bdgcmp with qpois or ppois) operate on comparably scaled signal tracks.

When NOT to use

• Input is already a deduplicated BAM or BED file (e.g., from a prior alignment pipeline); duplicate filtering would be redundant.
• Control sample is missing or the experiment is single-condition (unpaired); normalization cannot be computed without a baseline for depth correction.
• Fragment length d has not yet been estimated; scaling the lambda background requires knowledge of d to construct the pileup tracks at the correct extension length.

Inputs

• ChIP BED file (read locations, one per line: chromosome, start, end, etc.)
• Control BED file (same format as ChIP file)

Outputs

• Filtered ChIP BED file (duplicates removed)
• Filtered control BED file (duplicates removed)
• Final read count for ChIP sample (scalar: number of unique genomic positions after filtering)
• Final read count for control sample (scalar: number of unique genomic positions after filtering)
• Sequencing depth scaling factor (ratio: ChIP_reads / control_reads)

How to apply

First, filter duplicate reads from both ChIP and control BED files using macs3 filterdup with --keep-dup parameter (e.g., --keep-dup=1 keeps a maximum of 1 read per genomic location); record the final read counts for each sample, as these normalization factors are needed later. Second, compute the sequencing-depth scaling ratio as (final_ChIP_reads / final_control_reads) and apply it via macs3 bdgopt multiply when scaling the local lambda background track. This ensures that when ChIP and control pileup tracks are compared in bdgcmp, both are on the same effective sequencing depth scale, preventing false enrichment calls due to differential coverage rather than true ChIP signal.