clustering-accuracy-metric-extraction

clustering-accuracy-metric-extraction

Summary

Extract and tabulate clustering performance metrics (NMI, ARI, purity) from benchmark studies comparing single-cell ATAC-seq methods. This skill enables quantitative comparison of clustering quality across methods and datasets by compiling method-by-dataset matrices of standardized accuracy scores.

When to use

When you need to reproduce or validate benchmark comparisons between clustering methods on single-cell chromatin accessibility data, particularly when the source publication reports multiple accuracy metrics across heterogeneous datasets and you must decide which method variant (e.g., kmers+PCA vs. full-feature approaches) produces superior clustering.

When NOT to use

• If clustering metrics are not explicitly reported in the source study or supplementary materials—no extraction is possible without access to raw accuracy values.
• If the benchmark study lacks multiple independent datasets; single-dataset comparisons cannot be generalized using this skill's aggregation and ranking approach.
• If the study uses non-standard or undocumented clustering metrics that cannot be directly compared across methods without recalculation from raw cluster assignments.

Inputs

• Published benchmark results table or supplementary data file with method names, dataset identifiers, and accuracy metrics
• Method variant names and configurations (e.g., 'chromVAR kmers+PCA', 'SnapATAC with Leiden clustering')
• Clustering evaluation metrics (NMI, ARI, purity) as reported in the benchmark study

Outputs

• Method-by-dataset TSV matrix with rows as methods and columns as datasets, cells populated with accuracy scores
• Summary statistics table: mean, median, and per-method rank for each clustering accuracy metric across datasets
• Comparative finding document summarizing which method variant (e.g., kmers+PCA) is best within a framework and how it ranks against competing methods (e.g., SnapATAC)

How to apply

Locate the benchmark results table or supplementary data from the source publication (e.g., bioRxiv preprint 739011 for Chen et al.). Extract clustering accuracy metrics—NMI (normalized mutual information), ARI (adjusted Rand index), or purity scores—for each method variant across all reported datasets. Organize the metrics into a TSV table with rows as methods (e.g., chromVAR kmers+PCA, SnapATAC), columns as datasets, and cells as accuracy values. Calculate summary statistics (mean, median, per-method rank across datasets) to quantify relative performance and identify which method or variant achieves superior clustering fidelity. Document the metric definitions and any preprocessing or postprocessing steps (e.g., whether PCA was applied to the feature set before clustering) to ensure fair comparison.