Multi-Modal Interpretability

Advantage

By delegating the language side to a pre-trained vision-language model, multi-modal interpretability avoids the labor-intensive step of crowdsourcing per-neuron labels and scales to open-vocabulary concept sets — any text the alignment model recognises is a candidate label. The resulting concept-level explanations compose naturally with attribution: one can mask, swap, or steer a labelled neuron / SAE feature and observe the downstream effect on predictions, producing concept-conditional heatmaps and feature-visualisation videos that describe both what a unit represents and how it is used.

Limitation

Explanations inherit the biases and blind spots of the underlying alignment model: anything CLIP cannot embed cleanly (fine-grained categories, novel domains, low-resource languages, abstract qualifiers) yields noisy or empty concept rankings. Probing-image and concept-set selection also matter — different reference distributions can label the same neuron differently, so reported descriptions should always be read together with the concept set used. Finally, a high similarity score localises a concept correlationally; whether the unit is causally responsible for the model using that concept still requires interventional follow-up.

Submethods

The category typically takes two forms:

• Concept-Set Neuron Description (CLIP-Dissect): A scalable, training-free pipeline that automatically labels neurons in any vision DNN. For each neuron $u$, one collects its activations over a probing image set, summarises them into $\mathbf{a}_u$, and selects the concept $c \in \mathcal{C}$ whose CLIP text-embedding is most similar to the visual signature of those high-activating images. The resulting per-neuron descriptions cover the entire network — convolutional and transformer-based image classifiers alike — and let researchers compare neuron meanings across architectures, layers, and probing distributions. You can find a demo for this method in ./clip-dissect. This demo shows clip_dissect: Use this skill when you need to automatically describe or interpret the functionality of individual neurons in deep neural networks (DNNs) using CLIP-based semantic analysis, perform mechanistic interpretability research on vision models, dissect convolutional or transformer-based image classifiers, identify what visual concepts activate specific neurons, or compare neuron descriptions across different probing datasets and concept sets.