Sparse Autoencoders for Feature Discovery
A model has billions of neurons but they don’t map cleanly to concepts. Sparse autoencoders pull out cleaner features, one per discrete idea. The most concrete progress in interpretability.
The core idea
Train an autoencoder: a small network that compresses input to a hidden representation and reconstructs it. Add a sparsity penalty: most hidden units must be zero for any given input. Apply this to a transformer’s residual stream.
The result: the hidden units (“features”) tend to correspond to interpretable concepts. One fires for legal language, another for sad-sentiment text, another for Python list comprehensions.
Why it works
Raw transformer neurons are polysemantic: a single neuron fires for many unrelated concepts because the network packs more concepts into its representational space than it has neurons. SAE features are forced to be sparse, so each one specialises.
This is the “superposition hypothesis”: models compress concepts into shared neurons; the SAE projects them back into a wider, sparse representation where they’re separate.
Dictionary size and sparsity
An SAE has a hyperparameter: how many features (typically 8-1024 times the residual stream dimension). Larger dictionaries find more features but cost more compute. Sparsity (typical 30-100 active features per input) trades off interpretability vs reconstruction quality.
Anthropic’s scaled SAE work used millions of features per layer on Claude-class models. The result is a labelled dictionary of concepts the model has internalised.
Production uses (early)
- Steering: amplify or suppress a feature at inference time to nudge model behaviour without fine-tuning.
- Monitoring: alert when a “dangerous” feature activates strongly.
- Debugging: when a model misbehaves, inspect which features fired.
These applications are early-stage but fast-improving. By 2027, expect SAE-based monitoring to be a standard layer of LLM observability.