Dictionary learning and sparse autoencoders

Neurons mix many concepts. Superposition says why. This chapter shows the main tool that tries to undo superposition: a sparse autoencoder. We follow it from a 1996 neuroscience idea to 34 million features inside Claude 3 Sonnet, and then to the hard critiques of 2025.

Why we need a dictionary

Chapter 7 showed that you can read and write a concept, if you know which concept to look for. But how do you find concepts that nobody thought of? Chapter 3 ended with a problem. Many neurons are polysemantic: one neuron fires for cat faces, car fronts and some text. The superposition hypothesis explains this. The model wants to store more features than it has neurons. Most features are off most of the time. So the model packs them into directions that are almost, but not fully, at 90 degrees to each other.

If this is true, then the neuron is the wrong unit to study. The right units are the feature directions. But we do not know them. We only see the activation vectors. So we have a puzzle:

The list of directions is called a dictionary. Each direction is an atom (or "feature", or "latent"). The numbers f_i are the code. The code is sparse: most entries are zero. The dictionary is overcomplete: it has more atoms than the activation has dimensions. This is necessary, because superposition means there are more features than dimensions.

An old idea: sparse coding

Dictionary learning did not start with language models. It started in neuroscience and signal processing.

The objective in that paper is the same objective that an SAE uses: rebuild the input well, and use few active atoms. The difference is speed. Classic sparse coding solves a small optimization problem for each input to find its code. An SAE learns an encoder that guesses the code in one pass. The Anthropic team called this "amortized" sparse coding: you pay the cost once, during training.

Classic algorithms for this problem include K-SVD (2006) and online dictionary learning (Mairal et al., 2009). These give better codes for each input, but they do not scale to billions of activations and millions of atoms. SAEs give up a little quality to gain a lot of scale.

Words as sums of meanings

This was the SAE idea in small form, on static word vectors: one vector is a linear superposition of several meanings, and sparse coding can pull them apart.

Note the name Olshausen on both the 1996 paper and the 2021 paper. The line from neuroscience to language models is direct.

How a sparse autoencoder works

A sparse autoencoder is a small neural network with three parts. It reads one activation vector x from the model, for example the residual stream at one token. Let x have n numbers. The SAE has m hidden units, with m much larger than n.

1. Encoder. Subtract a bias. Multiply by a matrix W_enc. Add a bias. Apply ReLU. The result is the feature vector f, with m numbers. Most of them are zero.
2. Decoder. Multiply f by a matrix W_dec and add a bias. The result is x̂ ("x-hat"), the rebuilt activation.
3. Loss. Training punishes two things: the rebuild error, and the size of f.

Here d_i is column i of the decoder. Each decoder column is one feature direction. In the words of Chapter 3: d_i is the feature direction W_i, f_i is the feature activation x_i, and x is the activation vector. In the toy model of Chapter 3, we could read the directions from the model's own weights. A real model has no such matrix, so the SAE learns one. The rebuild is "a bias plus a few feature directions, each scaled by how strongly that feature is on".

The two parts of the loss pull in opposite directions:

• The reconstruction term ‖x − x̂‖² wants a perfect copy. It is happy to use many features.
• The sparsity term (the L1 penalty) adds up the feature strengths. It wants as few features on as possible.
• The number λ (lambda) sets the balance between them.

Why multiply by ‖d_i‖, the length of the decoder column? Without it, the SAE could cheat. It could make f_i tiny and make d_i huge, so the penalty would be near zero. The 2023 Anthropic paper stopped this by fixing every decoder column to length 1. Their 2024 update put the length into the penalty instead.

Some practical tricks

• Untied weights. The encoder and decoder are separate matrices. The 2023 paper found that the learned encoder and decoder differ in useful ways.
• Dead features. Some hidden units stop firing on every input. They waste space. Such units are called dead latents. The 2023 paper fixed this with resampling: from time to time, it moved each dead unit toward an input that the SAE rebuilt badly.
• Shrinkage. The L1 penalty pushes every active feature to be a little smaller than it should be. So the rebuild is too short. Later designs (TopK, Gated, JumpReLU) try to remove this effect.
• Normalization. Activations are scaled to a standard size before training, so that one value of λ means the same thing across layers.

Try it: train a toy SAE in your browser

This demo copies the logic of the first SAE experiments at a tiny scale. We create 2-dimensional data from 5 hidden directions (the grey rays). Five directions in two dimensions cannot be at 90 degrees to each other. This is superposition. Each data point turns on each hidden direction with probability p, with a random strength.

The SAE does not see the rays. It only sees the points. Press Train. The colored arrows are the SAE's decoder directions. Watch them turn and lock onto the rays.

What you see is the core claim of dictionary learning. If the data really is a sparse sum of directions, an SAE can find those directions without being told what they are. The "match" score is the mean, over true directions, of the best cosine similarity with any learned arrow. Sharkey and colleagues used the same score (they called it MMCS, mean max cosine similarity) in 2022.

Proof of concept: 2022 and 2023

This report used the same idea as the demo above, at a larger scale. It also asked: how do we know the right dictionary size? Their answer: as the size passes the true number of features, recovery stops improving, and dictionaries of nearby sizes start to agree. This report started the SAE agenda.

Cunningham et al. also did a causal test. On the indirect object identification task, they patched SAE features. They found that a smaller set of SAE features explained the behaviour than with other ways to split the activations. This was early evidence that SAE features are not only readable, but also useful units for circuits.

Towards Monosemanticity (Anthropic, 2023)

This paper made dictionary learning the central method of the field. The setup was simple on purpose:

• A one-layer transformer with a 512-neuron MLP, trained on 100 billion tokens of The Pile.
• SAEs trained on 8 billion MLP activation vectors.
• Dictionary sizes from 1× (512) to 256× (131,072). The main analysis used run "A/1" with 4,096 features.
• Feature names look like "A/1/3450": model A, run 1, feature 3450.

For each famous feature, the authors checked four things. Is it specific (when it fires, is the concept there)? Is it sensitive (when the concept is there, does it fire)? Does it cause the expected outputs? And is it absent from neurons (does no single neuron do the same job)?

What they measured

• Human scores. A blind rater scored random neurons and random features on a 0-to-14 rubric for how clearly they could be explained. The median neuron scored 0. The median feature scored 12.
• Automatic scores. Claude 2 wrote an explanation of each feature from examples. Then it predicted the feature's activation on new text from the explanation alone. Features scored much better than neurons.
• Loss recovered. They replaced the MLP output with the SAE's rebuild and measured the model's loss. For A/1 (4,096 features), the rebuild kept 79% of the MLP's benefit to the loss. The largest run (131,072 features) kept 94.5%.
• Dead features. In A/1, 168 of 4,096 features never fired on 100 million samples, and 292 more fired extremely rarely.

Feature splitting

When the dictionary grows, features do not simply get added. Existing features split into finer ones. In the 512-feature run there was one base64 feature. In the 4,096 run there were three. In the 16,384 run there were many. A general feature for "the" in mathematical text split into versions for machine learning ("the dataset", "the classifier"), abstract algebra ("the quotient", "the subgroup") and physics ("the gauge", "the Lagrangian").

The authors saw feature splitting as a feature, not a bug: different sizes give different zoom levels. But it also means the list of features is not unique. This point comes back in the critiques.

Other findings

• Universality. SAEs on two models trained with different random seeds (models A and B) found closely matching features.
• Feature "finite state automata". Some features chain together. Each one predicts tokens that turn on the next one. One such group produced valid HTML.
• Steering. Pinning the base64 feature high made the model write base64. Pinning the Arabic feature made it write Arabic.

Scaling Monosemanticity and Golden Gate Claude (2024)

Seven months later, the same team applied the method to Claude 3 Sonnet, a frontier model. Some choices changed:

• They read the residual stream at the middle layer, not an MLP. The residual stream is smaller, and middle layers seem to hold the most abstract features.
• They trained three SAEs: 1,048,576, 4,194,304 and 33,554,432 features.
• On average, fewer than 300 features were active per token. The rebuild explained at least 65% of the variance.
• Dead features grew with size: about 2% (1M), 35% (4M) and 65% (34M).
• Loss fell as a power law in compute, and the best number of features also grew as a power law. So SAE training can be planned with scaling laws, like model training.

What the features looked like

• Multilingual: one feature fires for the same concept in many languages.
• Multimodal: the SAE was trained only on text, but many features also fire on images of the same concept.
• Abstract: one feature fires on code with a security bug and on prose that discusses security flaws.
• Safety-relevant: features for unsafe code, backdoors, bias, sycophantic praise, deception, power-seeking, secrecy, and dangerous content. The authors warn: a feature that represents lying does not prove that the model lies.
• Feature neighbourhoods: features whose decoder directions are close tend to have related meanings. Near the Golden Gate Bridge feature are other San Francisco landmarks and Alcatraz. Further out are other bridges and famous places.

Feature completeness

Claude 3 Sonnet can name all the London boroughs. But only about 60% of them had their own feature in the 34M SAE. Whether a concept gets its own feature depends on how often it appears in training data. Rare concepts need much larger dictionaries. So the model holds many more features than even 34 million.

Golden Gate Claude

The most famous result is a steering experiment. Feature 34M/31164353 fires on the Golden Gate Bridge: on its name in many languages, on descriptions of it, and on pictures of it. The authors clamped this feature to 10 times its usual maximum value, on every token. Clamping means: run the SAE, set this one feature to a fixed value, decode, and put the result back into the model.

Clamping a "transit infrastructure" feature (1M/3) to 5× had a similar, smaller effect: the model mentioned a bridge where it normally would not. These steering results matter because they are causal. The features are not only correlated with concepts. Changing them changes what the model does.

OpenAI's TopK SAEs and scaling laws (2024)

OpenAI changed the activation function. A TopK SAE computes the encoder's pre-activations, keeps only the k largest, and sets all others to zero. There is no L1 penalty. The loss is only the rebuild error.

This has clear benefits. You set the sparsity directly: L0 is exactly k. There is no shrinkage, because nothing punishes the size of active features. And tuning is simpler.

Dead latents were a big problem at this scale. Without fixes, up to 90% of latents died. Two fixes brought this down to 7% in the 16M SAE:

• Start the encoder as the transpose of the decoder.
• Add an auxiliary loss (AuxK): the top dead latents must try to rebuild what the live latents missed. This gives dead latents a gradient and a job.

The paper also fit scaling laws: the rebuild error falls as a smooth function of the number of latents n and the sparsity k. It proposed metrics beyond rebuild error: the model's loss with the SAE spliced in, how well single latents act as probes for 61 known binary concepts, how well simple n-gram explanations predict latents, and how sparse each latent's effect on the output is.

Better SAEs: the architecture race

In 2024 and 2025 many groups proposed new SAE designs. Most try to fix two problems of the plain ReLU + L1 SAE: shrinkage, and a poor balance between sparsity and rebuild quality.

All these designs face the same basic trade-off. Fewer active features per token make each token easier to explain. More active features give a better rebuild. Researchers compare SAEs by plotting this trade-off curve, called the Pareto frontier. A better architecture pushes the curve up: a better rebuild at the same sparsity.

How do we grade an SAE?

We cannot compare SAE features to the "true" features, because nobody knows them. So the field uses several indirect scores.

Automatic interpretability

The method has three steps: explain (show an LLM examples where the unit fires and ask for a short explanation), simulate (give a second LLM only the explanation and new text, and ask it to predict the activation on each token), and score (correlate the predictions with the real activations). Auto-interpretability is the only way to label millions of features.

Simulation is expensive. In 2024, EleutherAI (Paulo, Mallen, Juang and Belrose) built a cheaper open pipeline that explains millions of features with open models. It added new scores: detection (given the explanation, pick which text snippets make the feature fire) and fuzzing (given highlighted tokens, decide if the highlighting is correct). They found SAE latents much easier to explain than neurons.

SAEBench

In March 2025, SAEBench (Karvonen and 14 co-authors) collected eight metrics in one benchmark and released over 200 SAEs across 8 architectures. Its main finding was uncomfortable: better scores on the standard proxy metrics (rebuild at a given L0) do not reliably mean better results on practical tasks. Matryoshka SAEs scored a little worse on the proxies but clearly better on tasks that need clean, separate concepts.

Open SAE suites and tools

Training large SAEs costs a lot. Open releases let many more researchers use them.

• Gemma Scope (Google DeepMind, August 2024): JumpReLU SAEs on every layer and sub-layer of Gemma 2 2B and 9B, plus some layers of 27B. More than 400 SAEs and more than 30 million features. Training used more than 20% of the training compute of GPT-3.
• Gemma Scope 2 (December 2025): tools for every layer of the Gemma 3 family (270M to 27B, base and instruction-tuned), including SAEs, transcoders (with and without a skip path), crosscoders for the base models, and cross-layer transcoders for the 270M and 1B models. The SAEs use Matryoshka training. Google DeepMind calls it the largest open release of interpretability tools by an AI lab so far.
• Llama Scope (Fudan / OpenMOSS, October 2024): 256 SAEs on every layer and sub-layer of Llama-3.1-8B.
• SAELens: an open Python library to train, load and study SAEs.
• Neuronpedia: a public website to browse features, read their top examples and explanations, steer with them, and (since 2025) explore attribution graphs.
• EleutherAI (sparsify, delphi), OpenAI (GPT-2 small SAEs), and Goodfire (SAEs for Llama 3.1 8B, Llama 3.3 70B and DeepSeek R1) also released SAEs and tools.

The problems with SAEs

From late 2024, careful studies found serious problems. None of them says that SAEs are useless. Together, they say that SAE features are not a clean list of the model's true parts.

1 · Feature absorption

Take two concepts: "starts with the letter S" (general) and "the token short" (specific). Every time short appears, both are true. An ideal dictionary would fire both features. But that costs two units of sparsity. The SAE can save one: it lets the "short" feature's direction also include the "starts with S" direction, and it turns the general feature off. The general feature now has a hole. It looks like a clean "starts with S" detector, but it silently fails on short.

The authors tested this with a spelling task on Gemma 2 2B and other models. They found absorption in hundreds of SAEs, and changing the SAE's size or sparsity did not fix it. This matters for safety. If you use an SAE feature as a detector, for example for "the model is thinking about deception", it may have holes you cannot see. Matryoshka SAEs reduce absorption, because their small, early dictionaries must contain the general features.

2 · Feature splitting and no canonical size

As we saw, larger SAEs split features into finer ones. A related 2024 study trained an SAE on the decoder directions of another SAE (a "meta-SAE"). It could break one latent into parts. So even a single latent is not always an atom. There is no single "right" dictionary.

3 · Dark matter

The dark matter of an SAE is the part of the activation it fails to rebuild: x − x̂. The study found that this error has structure. Larger SAEs fail mostly on the same inputs as smaller SAEs. So simply making the SAE bigger may not close the gap. Some of the missing part may be features that are not linear directions, such as the circular day-of-the-week features from Chapter 3.

4 · SAEs on random networks look good too

Heap, Lawson, Farnik and Aitchison (January 2025) trained SAEs on randomly initialized, untrained transformers. The auto-interp scores and rebuild metrics were similar to SAEs on trained models. So a good auto-interp score alone does not prove that the SAE found features that the model computes. The authors suggest that the input tokens themselves can make SAE latents look interpretable, even when the network has learned nothing. Targeted measures of how abstract a latent is can still separate trained from random models, so the scores are not meaningless. But studies now need a random-network baseline.

5 · Different seeds, different features

Paulo and Belrose (January 2025) trained SAEs that differed only in their random seed. For a 131K-latent SAE on Llama 3 8B, only about 30% of features were shared across seeds. ReLU + L1 SAEs were more stable than TopK SAEs. Their conclusion: treat SAE features as "a pragmatically useful decomposition", not as a complete list of the model's true features.

6 · Probes beat SAEs on known tasks

The DeepMind team wanted a practical test: detect harmful user intent in prompts to Gemma 2 9B IT, including in new styles of jailbreak and reworded prompts not seen in training. A dense linear probe on raw activations was almost perfect. A probe on one SAE feature could not even fit the training data. A probe on about 20 SAE features was fine in distribution, but clearly worse on new data. Probes trained on the SAE's rebuild did worse than probes on the raw activation. This means the SAE threw away information about the concept. SAEs trained on chat data closed about half the gap, but still lost. Their message: SAEs are still useful to explore and find unknown concepts, but if you already know the concept and have labels, a plain linear probe wins.

The team wrote that they were "deprioritising fundamental SAE research for the moment". Other studies agreed. AxBench (January 2025) found that SAEs did worse than simple baselines, such as prompting or difference-of-means directions, for detecting and steering concepts. A study of sparse probing over more than 100 datasets (Kantamneni, Engels and colleagues, February 2025) found that SAE probes did not reliably beat plain probes when data was scarce, labels were noisy, classes were imbalanced or the distribution shifted.

Where SAEs still help

The critiques did not end SAE research. They moved it toward the jobs where SAEs are strongest: discovering things you did not know to look for.

• Auditing hidden goals. Anthropic trained a model with a secret objective: to exploit known biases of its reward model. In a blind auditing game (Marks et al., March 2025), 3 of 4 teams found the hidden objective. SAE features were among the most useful tools. The team with only black-box access failed. (Chapter 11 has the full story.)
• Model diffing and emergent misalignment. OpenAI (Wang et al., June 2025) compared SAE features before and after a fine-tune that made a model broadly misaligned. They found "misaligned persona" features. One "toxic persona" feature controlled and predicted the misalignment. A few hundred good examples of further training restored alignment. (Chapter 11 tells the full story.)
• Dataset debugging. The DeepMind team found SAEs useful for spotting spurious correlations in data. (Kantamneni et al. found that simple non-SAE baselines could often do this too.)
• Pre-deployment audits. In the Claude Sonnet 4.5 system card (September 2025), Anthropic used SAE features such as "fake or suspicious content" to study and suppress evaluation awareness before release. (Chapter 11 has details on these system-card audits.)
• Hypothesis generation. A June 2025 position paper put it simply: use SAEs to discover unknowns, not to act on knowns. If you already know the concept you want, train a probe. If you do not know what to look for, an SAE can show you.

Beyond SAEs: the next tools

SAEs describe what is in an activation. They do not directly say how the model computes it. To trace computation, researchers built close relatives of the SAE. Chapter 9 covers them in detail:

• Transcoders (Dunefsky, Chlenski, Nanda, 2024) read the input of an MLP and predict its output with sparse features. This makes it simple to connect features across a layer.
• Crosscoders (Anthropic, 2024) read and write several layers, or several models, at once. They help with features smeared across layers and with comparing a base model to its chat version.
• Cross-layer transcoders (Anthropic, 2025) power the attribution graphs that traced multi-step reasoning and rhyme planning in Claude 3.5 Haiku.
• Parameter decomposition (APD and SPD, Apollo, 2025) splits the weights instead of the activations into simple pieces.
• Weight-sparse transformers (OpenAI, November 2025) train models whose weights are almost all zero, so circuits are small from the start.

References

1. Olshausen, B. & Field, D. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature.
2. Arora, S. et al. (2018). Linear Algebraic Structure of Word Senses, with Applications to Polysemy. TACL.
3. Yun, Z., Chen, Y., Olshausen, B., LeCun, Y. (2021). Transformer visualization via dictionary learning.
4. Elhage, N. et al. (2022). Toy Models of Superposition.
5. Sharkey, L., Braun, D., Millidge, B. (2022). Taking features out of superposition with sparse autoencoders.
6. Bills, S. et al. (2023). Language models can explain neurons in language models. OpenAI.
7. Cunningham, H. et al. (2023). Sparse Autoencoders Find Highly Interpretable Features in Language Models.
8. Bricken, T. et al. (2023). Towards Monosemanticity. Anthropic.
9. Anthropic (2024). Circuits Updates — April 2024 (SAE training update).
10. Templeton, A. et al. (2024). Scaling Monosemanticity. Anthropic.
11. Anthropic (2024). Golden Gate Claude.
12. Gao, L. et al. (2024). Scaling and evaluating sparse autoencoders. OpenAI.
13. Rajamanoharan, S. et al. (2024). Improving Dictionary Learning with Gated Sparse Autoencoders.
14. Rajamanoharan, S. et al. (2024). Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders.
15. Bussmann, B., Leask, P., Nanda, N. (2024). BatchTopK Sparse Autoencoders.
16. Bussmann, B. et al. (2025). Learning Multi-Level Features with Matryoshka Sparse Autoencoders.
17. Braun, D. et al. (2024). Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning.
18. Lieberum, T. et al. (2024). Gemma Scope. Google DeepMind. · Gemma Scope 2 docs.
19. He, Z. et al. (2024). Llama Scope.
20. SAELens · Neuronpedia · EleutherAI sparsify.
21. Paulo, G. et al. (2024). Automatically Interpreting Millions of Features in Large Language Models.
22. Karvonen, A. et al. (2025). SAEBench. ICML 2025.
23. Chanin, D. et al. (2024). A is for Absorption. NeurIPS 2025.
24. Engels, J., Riggs, L., Tegmark, M. (2024). Decomposing The Dark Matter of Sparse Autoencoders. TMLR.
25. Heap, T. et al. (2025). Automated Interpretability Metrics Do Not Distinguish Trained and Random Transformers. (first titled "Sparse Autoencoders Can Interpret Randomly Initialized Transformers")
26. Paulo, G. & Belrose, N. (2025). Sparse Autoencoders Trained on the Same Data Learn Different Features.
27. Smith, L. et al. (2025). Negative Results for SAEs On Downstream Tasks. Google DeepMind.
28. Marks, S. et al. (2025). Auditing Language Models for Hidden Objectives. Anthropic.
29. Wang, M. et al. (2025). Persona Features Control Emergent Misalignment. OpenAI.
30. Peng et al. (2025). Position: Use Sparse Autoencoders to Discover Unknowns. ICML 2026 (first titled "Use Sparse Autoencoders to Discover Unknown Concepts, Not to Act on Known Concepts").
31. Dunefsky, J., Chlenski, P., Nanda, N. (2024). Transcoders Find Interpretable LLM Feature Circuits.
32. Lindsey, J. et al. (2024). Sparse Crosscoders for Cross-Layer Features and Model Diffing.
33. Ameisen, E. et al. (2025). Circuit Tracing. Anthropic.