SAE Interpretability Evaluation: The Measurement Pipeline Matters More Than the Model Architecture

In July 2026, researchers published a paper on arXiv showing that when sparse autoencoders (SAEs) are compared on interpretability, the result depends more on the chosen evaluation pipeline than on the architecture of the models themselves.

What was tested and how

The authors tested whether autointerpretability scores capture stable properties of a model's features or merely side effects of the measurement procedure. In the standard pipeline, one language model explains each SAE feature, and a second language model scores the quality of that explanation. These scores are what researchers rely on when they compare the interpretability of SAEs across different papers.

The experiment covered four metrics, two models and four axes of methodological variation. Across all conditions, the assumption that the scores are stable did not hold.

• Four evaluation metrics: simulation, detection, fuzzing and purity
• Two models: Pythia-160M and Apertus-8B
• Four axes of methodological variation in the evaluation pipeline
• Detection is the most stable metric; fuzzing is unreliable under every condition
• Three tools from the authors: variance decomposition, a Stability Check and a Minimum Reporting Checklist

Why methodology matters more than architecture

The paper's main finding is that methodological variability, taken together, exceeds architectural variability on every metric and for both models tested. Put simply, the spread in scores caused by how the measurement is set up turned out to be larger than the spread between the different SAE architectures that the measurement is supposed to tell apart.

"Methodological variability in aggregate exceeds architectural

variability across all metrics and all tested models," the paper's abstract on arXiv states.

This undermines the logic of cross-paper comparisons. If two teams report different autointerpretability scores, the gap may come from how their pipelines are configured rather than from the quality of the autoencoders.

Where the scores break down

Each metric showed its own instability profile. Detection held up best; fuzzing, by contrast, was unreliable under every condition checked. The metrics are not interchangeable: the choice of metric alone changes the final conclusion.

Feature ranking is a separate problem. According to the study, top-k lists of the most interpretable features do not persist when the corpus or sampling conditions change. Meanwhile, average scores remain stable and mask the instability at the level of individual features. This is a failure that cannot be caught by tracking explanation similarity alone.

What this means

Unreliable evaluation slows progress in interpretability at exactly the moment when reliable tools for understanding AI systems are most needed. So that comparisons in the SAE literature stop reflecting pipeline differences instead of model differences, the authors propose a variance decomposition method, a Stability Check procedure and a Minimum Reporting Checklist, a minimal set of parameters that papers should disclose.

Frequently asked questions

Which evaluation metric proved the most reliable?

According to the paper, detection was the most stable of the four metrics. Fuzzing, by contrast, was found to be unreliable under every condition tested, while simulation and purity each showed their own instability profiles.

What do the authors propose for more reliable evaluation?

The authors propose three tools: a variance decomposition method, a Stability Check procedure for testing how robust the scores are, and a Minimum Reporting Checklist, a list of pipeline parameters that papers should disclose so that comparisons stay comparable.