Oxford Releases a 180-Page Neural Network Interpretability Textbook With Code

We have long grown used to a strange situation: a model answers correctly, and nobody knows why it answers that way. Not the developer, not the product manager, and sometimes not even the authors of the model itself. When it suddenly starts flattering the user, inventing facts, or bypassing restrictions, we mostly treat the symptom: tweak the prompt, fine-tune, add a filter. Mechanistic interpretability offers a different approach: open the model up and look at which internal mechanisms are responsible.

Researchers from the University of Oxford and other scientific organizations have published a large 180-page survey that gathers this field into a single textbook. It is oriented toward AI safety. It comes with an interactive course and Jupyter Notebooks containing hands-on experiments.

What's inside the textbook

According to the authors, the material is built around four blocks.

The first is superposition and transformer circuits. The idea is that a network has fewer neurons than concepts it needs to store. As a result, a single neuron often participates in several meanings at once, and an individual neuron tells you little. This section explains how networks encode information and how computation emerges from that.

The second is sparse autoencoders (SAEs). This is a method that takes a model's internal representations and decomposes them into cleaner "features." Instead of a tangled activation vector, you get a set of understandable directions, roughly "this is about code" or "this is about apologies." Such features can already be searched for, compared, and tracked.

The third is circuit discovery. It deals with how to find a small part of the network responsible for a specific behavior, and verify that it really is responsible, rather than merely coinciding in time.

The fourth is an applied safety section: steering (controlling model behavior), knowledge editing, and monitoring hidden mechanisms.

Why it's worth opening the practical part

Usually interpretability looks like a set of papers that you have to piece together bit by bit. Here there is a unified structure and code you can run and poke at. For an engineer, this lowers the barrier to entry: you can take a small open model, run an experiment from a notebook, and see what a "feature" looks like inside a network.

Here is where this can be useful in practice:

• If you deploy open-weights models, you have access to activations, which means methods like SAEs and steering are available to you. With closed APIs you usually don't have such access, and that is an important limitation.
• If you are responsible for product quality or safety, monitoring internal features can complement external checks such as tests and filters. This is an idea, not a ready-made recipe, and the textbook shows where to start.
• If you are a product manager or founder, the course will help you understand what interpretability researchers are actually promising and where reality ends.

Limitations the authors themselves point out

The main thing worth noting: the survey does not sell interpretability as a magic button. The authors directly address the limitations of existing methods and stress that understanding internal mechanisms does not by itself guarantee a model's safety. Seeing that a network has a "deception feature" is not the same as reliably switching it off and being sure the behavior won't return by another path.

Moreover, this is a textbook and a survey, not a new tool that will slot into your pipeline tomorrow. If you are looking for a ready-made production solution, this isn't it. But it is a map of the territory, and it is handy for finding your bearings when the next release claims that "we have learned to understand how the model thinks."

What to do

If you work with models at a level deeper than the API, save the links to the arXiv preprint 2611.08438 and the interactive course. Set aside an evening for the notebooks and try at least one SAE or steering experiment on a small model. Even if it never makes it into your product, you will start to better understand what is happening inside your models. The black box remains a black box, but now it comes with instructions for opening it.