compressed-sparse-row-matrix-handling

compressed-sparse-row-matrix-handling

Summary

Convert and initialize single-cell count matrices into compressed sparse row (CSR) format compatible with matrix-free spectral embedding and scalable dimension reduction algorithms. CSR representation is essential for memory-efficient processing of large sparse single-cell datasets (10M+ cells) in downstream spectral and clustering workflows.

When to use

You have a raw or preprocessed single-cell count matrix (from BAM-to-fragment or FASTQ-to-matrix pipelines) and need to apply matrix-free algorithms like tl.spectral, tl.multi_spectral, or other scalable dimension reduction methods that require dense or sparse matrix input. Use CSR format specifically when dataset size exceeds typical in-memory dense matrix limits (e.g., >1M cells) or when downstream tools (SnapATAC2, Scanpy) explicitly expect sparse backends.

When NOT to use

• Input matrix is already in CSR or other optimized sparse format (e.g., scipy.sparse.csc_matrix) — skip conversion and proceed directly to downstream tools.
• Dataset size is <100K cells and memory is not a constraint — dense matrix storage may be simpler and faster for small-scale exploratory analysis.
• Downstream tool explicitly requires dense matrix input (rare for SnapATAC2 ecosystem, but check tool documentation).

Inputs

• count matrix (tile-based, peak-based, or gene-based quantification)
• cell-by-feature matrix in dense array format, scipy.sparse format, or HDF5
• AnnData object with .X as dense array

Outputs

• AnnData object with .X as CSR-backed sparse matrix
• scipy.sparse.csr_matrix
• compressed sparse row representation compatible with tl.spectral

How to apply

Load the count matrix (tile-based, peak-based, or gene-based) from its native format (HDF5, AnnData, or text-based sparse formats) and convert to CSR format using SnapATAC2's matrix utilities or scipy.sparse. Initialize the matrix in SnapATAC2's AnnData object with .X as a CSR-backed array; this enables lazy evaluation and reduces peak memory footprint during subsequent operations. Verify CSR integrity by checking matrix shape, sparsity (number of nonzero elements), and data type consistency. For datasets approaching or exceeding 10 million cells, confirm that memory profiling shows sub-linear memory growth relative to cell count when CSR is used versus dense alternatives.