anndata-backed-object-manipulation

anndata-backed-object-manipulation

Summary

Manipulate fully backed AnnData objects to store and access single-cell omics fragment data and count matrices without loading entire datasets into memory. This skill enables scalable preprocessing and matrix operations on datasets exceeding 10 million cells by leveraging on-disk storage.

When to use

When working with large single-cell ATAC-seq or multi-omics datasets where in-memory storage is infeasible (>1M cells), and you need to iteratively add or modify count matrices (tile-based, peak-based, or gene-based) while preserving fragment-level data for reproducibility and re-analysis.

When NOT to use

• Input data is already a dense in-memory count matrix; use backed AnnData only to avoid reprocessing raw fragments.
• Workflow requires frequent random access to all cells × features values (backed mode has I/O latency); consider in-memory AnnData for small datasets (<1M cells).
• Fragment data is unavailable or lost; count matrices cannot be regenerated with alternative binning strategies without raw sequencing alignments.

Inputs

• backed AnnData object (.h5ad file on disk)
• fragment coordinate data stored in .obsm slots (paired-end or single-end fragment tuples)
• genomic interval definitions (tile coordinates, peak BED file, or gene GTF annotations)

Outputs

• count matrix (stored in .X or .obsm of the same backed AnnData object)
• updated backed AnnData object with added matrix layer
• metadata about matrix binning (tile size, peak list, gene annotations)

How to apply

Load or create a backed AnnData object using SnapATAC2's I/O functions, which stores dense arrays and sparse matrices on disk rather than in RAM. Store raw fragment coordinates in .obsm['fragment_paired'] or .obsm['fragment_single'] slots. Apply matrix operations (pp.add_tile_matrix, pp.make_peak_matrix, pp.make_gene_matrix) to generate count matrices from fragments; each operation reads from and writes to disk without materializing the full dataset. Verify matrix shape (n_obs × n_vars) and sparsity after each operation to confirm the count matrix reflects the intended genomic binning or feature selection.