Chapter 12 · Reference

Timeline, glossary, reading list

This chapter is a toolbox for the whole survey. Use it to see how the ideas connect, to find a date, to look up a word, or to choose what to read next.

On this page

  • Concept map. Click an idea to see what it builds on and what it led to.
  • Full timeline. About 130 dated events from 1984 to September 2026. Filter by topic or chapter, or search.
  • Glossary. About 185 terms in simple English. Type to search.
  • Reading list. A staged path (beginner → core → advanced → newest) with a chapter link for every item, plus courses and tools.
  • Notation, about this survey, and a final test yourself quiz.

How the ideas connect

The field did not grow in a straight line. Some ideas motivated others: polysemantic neurons motivated the study of superposition, and superposition motivated sparse autoencoders. Some ideas built on others: transcoders built on SAEs, and attribution graphs built on transcoders. Some methods test the claims of others: patching tests circuit hypotheses.

The map below shows these links. Columns go roughly from older ideas (left) to newer ideas (right). The last column holds what became important in 2026: white-box audits in system cards, natural language autoencoders, probes in production, and personas. Click a box to highlight its links and read a short summary.

Interactive · Concept map of mechanistic interpretability
Click any box. Click "Show all" to clear the highlight. On a small screen, scroll the map sideways.
builds on motivates tests
Four threads to follow

The "what" thread (top of the map) asks what a model represents: feature visualization → polysemanticity → superposition → sparse autoencoders → self-explaining models → natural language autoencoders.

The "how" thread (bottom) asks how parts compute: residual stream → attention heads → induction heads → hand-found circuits → attribution graphs.

The "use" thread (right) asks what the tools are good for: directions and probes → probes in production; auditing games → white-box audits of real models before release.

The "prove it" thread (bottom left) asks which parts cause behaviour: ablation → activation patching → path patching → causal scrubbing, ACDC and DAS. The strongest results use several threads at once.

Full timeline

The band shows each event as a dot, coloured by topic. Hover a dot to read it. Click a dot to jump to it in the list. Use the buttons, the chapter menu or the search box to filter both views.

Interactive · The field, 1984–2026 (to September 2026)
Hover a dot to read the event.

Five eras stand out:

  1. Roots (before 2013). Mathematics that later explained superposition (the Johnson–Lindenstrauss lemma, compressed sensing) and sparse coding of images.
  2. Seeing inside vision models (2013–2019). Saliency maps, feature visualization, Network Dissection and TCAV. People asked "what does this unit detect?"
  3. Circuits (2020–2021). The Distill Circuits thread and the first mathematical framework for transformers. People asked "how do units connect?"
  4. LM circuits and superposition (2022–2023). Induction heads, IOI, grokking, toy models of superposition, patching methods and the first SAEs.
  5. Scale, replacement models and use (2024–2026). SAEs on frontier models, attribution graphs, self-explaining models (up to natural language autoencoders), probes in production, and white-box audits of real models before release.

Glossary

Each term has a short definition in simple English. Many of these words also appear underlined in the chapters. Hover them there to see the same definition.

Interactive · Searchable glossary

Reading list

This path goes from easy to hard, in four stages. You do not need to read everything. Read the beginner stage first, then pick the papers that match your interest. In stages 2 and 3, items follow the order of the chapters; stage 4 is in date order. Each entry links to the chapter of this survey that explains it.

1

Beginner: start here

Short, friendly pieces. A weekend of reading.

2

Core papers

The classic results that every later paper builds on. Read with Chapters 2–8.

3

Advanced methods

Rigorous causal tools, SAEs at scale, transcoders and other threads.

4

Newest work (2025–2026)

In date order. These change fast; check for newer versions.

Courses, tools and places to read

ARENA

Free hands-on notebooks: TransformerLens, induction heads, IOI, superposition, SAEs, Othello-GPT.

Neel Nanda's guides

Getting started, an explainer and glossary, open problems, and an annotated paper list.

TransformerLens

A library to load open models, cache every activation, and add hooks for patching.

nnsight / NDIF

Write interventions on any PyTorch model, and run them on very large models remotely.

SAELens

Train, load and analyse sparse autoencoders. Many pretrained SAEs are available.

Neuronpedia

Browse SAE and transcoder features (including Gemma Scope 2), try steering, and explore attribution graphs in your browser.

circuit-tracer

Open-source code to build attribution graphs for open models such as Gemma 2, Llama 3.2 and Qwen3.

Transformer Circuits Thread

Anthropic's papers, essays and circuits updates, from the 2021 framework to NLAs and the global workspace.

Distill (archive)

The vision-era classics: feature visualization, building blocks, the Circuits thread.

Alignment Forum

Where many results first appear: Google DeepMind updates, MATS projects, critiques and replications.

MATS

A mentorship program that has trained many mech-interp researchers.
Links change

Blog posts and project pages sometimes move. If a link breaks, search for the exact title. The arXiv links are the most stable.

Notation cheat sheet

The chapters use the symbols below. They follow the conventions of "A Mathematical Framework for Transformer Circuits" and the main SAE papers.

SymbolMeaningWhere
xAn activation vector (for example, the residual stream at one token).everywhere
d, d_modelThe number of dimensions of the residual stream (768 in GPT-2 small).Ch. 3, 4
W_EEmbedding matrix: token → vector.Ch. 4
W_UUnembedding matrix: vector → logits over the vocabulary.Ch. 4
W_Q, W_K, W_VQuery, key and value matrices of one attention head.Ch. 4
W_OOutput matrix of a head: writes its result back to the residual stream.Ch. 4
W_QK = W_Qᵀ W_KThe QK circuit: decides where the head attends.Ch. 4
W_OV = W_O W_VThe OV circuit: decides what the head copies.Ch. 4
A, A^hAttention pattern (who attends to whom) of head h.Ch. 4
L.H (e.g. 9.9)Head H in layer L, both counted from 0.Ch. 5, 6
n, mToy models: number of features n, number of hidden dimensions m (m < n).Ch. 3
SSparsity: the chance that a feature is zero on an input.Ch. 3
IᵢImportance of feature i in the toy-model loss.Ch. 3
W, bToy model: x' = ReLU(WᵀW x + b).Ch. 3
fSAE feature activations (a long, mostly-zero vector).Ch. 8
W_enc, b_encSAE encoder weights and bias: f = ReLU(W_enc(x − b_dec) + b_enc).Ch. 8
W_dec, b_decSAE decoder: x̂ = W_dec f + b_dec. Each column is one feature direction.Ch. 8
x̂The reconstruction of x.Ch. 8, 9
λSparsity coefficient: the weight of the L1 penalty. (In Ch. 10, λ is the learning coefficient.)Ch. 8, 10
L0Average number of active SAE features per token.Ch. 8
kTopK SAE: the number of features kept per input.Ch. 8
LDLogit difference: logit(correct) − logit(wrong).Ch. 5
αSteering coefficient: how much of a direction to add.Ch. 7
v, r̂A steering or concept direction; r̂ is the unit refusal direction.Ch. 7

Test yourself

These questions cover the whole survey. If you get one wrong, the explanation points to the chapter to reread.

Why do many neurons respond to several unrelated concepts?
Superposition lets a model pack many sparse features into fewer dimensions. Neurons then mix features. See Chapter 3.
An induction head predicts [B] after seeing [A][B] … [A]. Which earlier head does it need?
Induction heads use K-composition with a previous-token head. See Chapter 4.
In activation patching, you run the corrupted prompt and copy in one clean activation. The output becomes almost clean. What does that show?
This is denoising. It tests sufficiency. Noising (the opposite direction) tests necessity. See Chapter 5.
A linear probe reads "is this statement true?" from layer 12 with high accuracy. What should you do next to show the model uses this information?
Probes show presence. Interventions show use. See Chapter 7.
What is the main trade-off when you train a sparse autoencoder?
A stronger sparsity penalty gives fewer active features but worse reconstruction. See Chapter 8.
Why did Anthropic use cross-layer transcoders instead of plain SAEs to build attribution graphs?
SAE features sit on activations, but the MLP between them is still a black box. Transcoders replace that computation. See Chapter 9.
In the "Biology of a Large Language Model" poem example, what showed that Claude plans ahead?
The plan features appeared at the newline before the line. Suppressing or replacing them changed how the line ended. See Chapter 9.
For "the capital of the state containing Dallas is", the attribution graph shows Dallas → Texas → say "Austin". What happened when researchers suppressed the "Texas" features?
Without "Texas", the model no longer knew which capital to say, so it spread its bets over generic words. Other capitals (Albany, Sacramento) appeared only when the "Dallas" features were suppressed instead. Swapping "Texas" for "California" gave "Sacramento". See Chapter 9.
Which statement best describes the state of the field in 2026?
The field has real tools and real uses (white-box audits in system cards, probes in production), and also open problems and active debate (attribution graphs explain only some prompts; probes can be evaded; SAE versus simpler methods). See Chapter 11.

About this survey

How it was made

Limits

Where to go from here

The best way to learn mechanistic interpretability is to do it. Pick a small open model, load it with TransformerLens or nnsight, and reproduce one result from this survey, such as induction heads or the IOI patching map. Then change one thing and see what happens.