deepspot-m

DeepSpot-M

Overview

DeepSpot-M is a multimodal foundation model that maps a 224x224 H&E histology tile to spatial gene expression in log1p-CPM. The output is virtual spatial transcriptomics: one value per queried gene per tile, laid out on the grid the tiles came from.

A LoRA-adapted pathology foundation backbone (Midnight) tokenises the tile. A cross-attention gene decoder lets each gene query attend to the patch tokens, and a gene router hypernetwork builds gene-specific projections from frozen biological embeddings (Evo 2, Orthrus, ProtT5, scGPT, Apertus). Genes enter the model as queryable embeddings rather than fixed output slots, so the released model covers a ~19k protein-coding gene panel including genes unseen in training. The panel ships with the weights as tokens.csv and is exposed as model.gene_names; genes outside it cannot be queried in this release.

Applied to TCGA, the model produced a virtual spatial transcriptomics atlas of 28,664 slides across 32 cancer types.

Licensing

The code is PolyForm Noncommercial 1.0.0 and the weights are CC-BY-NC-SA-4.0. Use it for noncommercial research and check both licences before redistributing outputs.

Installation

uv pip install deepspotm==1.0.0

Version 1.0.0 targets Python 3.10 to 3.13 and pulls in PyTorch. Install the PyTorch build that matches your CUDA version first if you want GPU inference.

Model access

The weights are gated:

1. Open https://huggingface.co/ratschlab/DeepSpotM and request access.
2. Once access is granted, authenticate the machine that will download them:

huggingface-cli login

from_pretrained reads that cached token, so a login is needed once per machine.

Quick start

from deepspotm import DeepSpotM

model, image_processor = DeepSpotM.from_pretrained("ratschlab/DeepSpotM", source="scgpt")

vals = model.predict_genes(image_processor(pil_tile).unsqueeze(0), ["EPCAM", "CD3D"])

pil_tile is a PIL image of exactly 224x224 pixels. image_processor turns it into a tensor, unsqueeze(0) adds the batch dimension, and predict_genes takes the batch plus a list of HGNC gene symbols. Values come back in log1p-CPM, aligned with the gene list you passed, so keep that list beside the output to keep the columns labelled. Symbols must be in the released ~19k-gene panel (model.gene_names); an unknown symbol raises KeyError naming the offending genes.