ica-lens

ICA Lens: A Training-Free Interpretability Lens on Model Components

When to Use

Activate this skill when the goal is to characterise the interpretable structure of a component's activation distribution without paying the up-front cost of training a sparse dictionary.

Typical triggers:

• A new model, layer, or hook point needs an initial inventory of candidate interpretable directions before committing to SAE / transcoder training.
• An existing SAE / transcoder feature set needs an independent baseline of interpretable directions to compare against (overlap, coverage, perturbation strength, sparse-probe accuracy under a matched budget).
• A specific concept or behaviour is suspected to live on a low-dimensional, non-Gaussian subspace of a residual / MLP / head-output state, and a fast, gradient-free extractor is needed to expose it.
• A study of non-Gaussianity itself — where interpretable structure concentrates across depth, modality, and model family — is the object of interest.
• Targeted perturbation / steering needs a direction handle whose discovery is decoupled from the model's training signal.
• An interpretability artefact needs to be browsed, annotated, or shared via an explorer UI rather than recomputed each session.

Keywords: ICA, Independent Component Analysis, FastICA, non-Gaussianity, dictionary-free interpretability, interpretable directions, component interpretability, residual-stream directions, MLP-output directions, attention-head directions, sparse probing, targeted probe perturbation, SAE alternative, SAE baseline, ICA explorer.

Method in One Paragraph

The ICA Lens treats each cached activation $\mathbf{a} \in \mathbb{R}^{d_{\text{model}}}$ at a chosen hook point as a sample from an unknown source distribution and fits an unmixing matrix $\mathbf{W}{\mathrm{ICA}}$ such that the components $\mathbf{f}(\mathbf{a}) = \mathbf{W}{\mathrm{ICA}},\mathbf{a}$ are maximally non-Gaussian and mutually independent. The motivating observation is that interpretable, token-selective directions in language models appear systematically less Gaussian than random directions in the same space, so non-Gaussianity is a sufficient surrogate objective to recover candidate monosemantic axes without any reconstruction or sparsity loss. Compared with sparse-autoencoder-family methods, no over-complete dictionary is trained, no per-feature dead-direction / split-feature pathology is incurred, and the recovered directions can be evaluated by exactly the same downstream protocols — sparse probing, targeted probe perturbation, top-activating example mining, and feature dashboards.