Inside the Black Box
A plain-language guide to mechanistic interpretability: the science of reverse-engineering neural networks. It goes from the first saliency maps in 2013 to attribution graphs of frontier language models today.
The story in five sentences
- Neural networks learn their own algorithms during training, and nobody writes those algorithms down. Mechanistic interpretability tries to reverse engineer them.
- Researchers first looked inside vision models (2013–2021). They found curve detectors, dog-head detectors and real circuits, read straight from the weights.
- They also found that many neurons mix unrelated concepts, because networks pack more features than they have neurons (superposition). In transformers, they tested explanations with causal tools such as activation patching.
- Sparse autoencoders (2023–2024) unpacked superposition into millions of readable features, and attribution graphs (2025) traced how Claude plans rhymes, does arithmetic and refuses requests.
- Since 2025 the field has turned practical: white-box audits in model system cards, cheap probes in production, models that explain their own activations in words, and a debate on whether to aim for full understanding or for simple tools that work.
Four eras of the field
Seeing inside vision models
Saliency maps, feature visualization, DeepDream, Network Dissection, TCAV, and the first readable neurons in text models.
The Circuits program
"Zoom In", curve detectors, multimodal neurons, the name "superposition", and the first transformer math framework.
Circuits in language models
Induction heads, IOI, grokking, superposition toy models, activation patching, steering vectors, the first sparse autoencoders.
Scale, use and a pragmatic turn
SAEs on frontier models (Golden Gate Claude), attribution graphs, white-box audits in system cards (from Sept 2025), probes in production (Jan 2026), natural language autoencoders (2026), and "method minimalism": test against simple baselines first.
Chapters
Interactive timeline
Filter by topic. Each entry links to the chapter that explains it.
Suggested learning paths
Times are estimates: reading at about 200 words per minute, plus about 3 minutes for each interactive widget. Every chapter starts with a "Learning path" box (what you need first, what you will learn) and ends with "Key takeaways".
From the birth to 2026
1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 → 9 → 10 → 11 → 12. The chapters follow the history, and each one builds on the one before it. Spread it over a week: about one chapter per sitting.
You already know transformers
Skim 1 (10 min) → 3 (45) → 4, from the residual stream to induction heads (25) → 5 (50) → 8 (45) → 9 (45) → 11 (55).
Audits, monitors and hidden goals
1 (15 min) → 3 (45) → 7 (65) → 8, problems and uses of SAEs (20) → 9, attribution graphs and case studies (35) → 10, self-explanation and personas (30) → 11 (55).
Start with pictures
1 (15 min) → 2 (50) → 3 (45) → 8 (45) → 10, CLIP, ViTs, image generators and biology models (25) → 11, the field in 2026 (15). Add 4 (40) to read the transformer work.
| Chapter | Words | Widgets | Time |
|---|---|---|---|
| 1 · What is mech interp? | ~2,000 | 1 | ~15 min |
| 2 · The birth: vision models | ~8,000 | 4 | ~50 min |
| 3 · Features and superposition | ~7,000 | 4 | ~45 min |
| 4 · Transformers | ~5,200 | 4 | ~40 min |
| 5 · Causal tools | ~7,000 | 5 | ~50 min |
| 6 · Circuit case studies | ~7,300 | 4 | ~50 min |
| 7 · Probing and steering | ~11,000 | 3 | ~65 min |
| 8 · Dictionary learning | ~6,700 | 3 | ~45 min |
| 9 · Circuit tracing | ~7,300 | 3 | ~45 min |
| 10 · New directions | ~8,800 | 4 | ~55 min |
| 11 · Uses, debates, frontier | ~7,700 | 6 | ~55 min |
| 12 · Reference | ~2,700 | — | ~20 min (then use as a lookup) |
How to read this
- New to the field? Take the complete course, in order. Do the widgets and quizzes; they are part of the lesson.
- Know transformers already? Take the fast track above.
- Want the newest results? Go to chapters 9, 10 and 11. Each starts with a list of what to read first.
- Want a quick reference? Chapter 12 has the glossary, a full timeline and a reading list.
This survey summarizes published papers and research posts. Where we give a number or a model name, it comes from the original source, which is linked. Simple language means we must sometimes leave out detail. When in doubt, read the paper.