SAE on Gemma-2-2B: Share of Matched Features Rises from 2% to 29% With 50M Tokens

Sparse autoencoders (SAE) have become one of the main tools for interpreting language models, and they are typically trained on hundreds of millions or billions of activation tokens. Almost no one has checked whether that scale is actually necessary. The authors of a new paper from Hugging Face Daily Papers did: they took four SAE architectures, three language models, and budgets ranging from 5M tokens up to a 500M reference. The main result is that the answer depends on what exactly you measure. For Gemma-2-2B with BatchTopK, the share of strictly matched features rises from 2% at 5M tokens to 29% at 50M.

Two Measures of Convergence

The researchers separated two concepts. The first is functional behavior: reconstruction quality and results on standard SAE benchmarks. The second is one-to-one reproducibility: whether an independently trained SAE finds the same features as the reference one.

The gap turned out to be significant. Reconstruction often reaches the level of the full reference at much smaller budgets. Feature reproducibility, meanwhile, keeps improving as data grows. An SAE can look "ready" by the metrics while its feature dictionary remains unstable.

Key findings from the paper:

• Scale of the experiment: 4 SAE architectures, 3 language models, budgets from 5M to 500M tokens.
• Gemma-2-2B, BatchTopK: 2% of features strictly matched at 5M tokens and 29% at 50M.
• Data requirements depend heavily on the metric: reconstruction saturates earlier, reproducibility does not.
• Random sampling better preserves the original distribution of activations and usually wins on reconstruction.
• Diversity-based sampling often improves AutoInterp results.

How to Choose Your Data

The volume of data matters, but so does how you select it. Random sampling reproduces the activation distribution more accurately, so it is usually preferable when good reconstruction is the goal. Diversity-based sampling covers more of the model's rare behavioral modes, which often raises AutoInterp scores. AutoInterp is automated interpretation of features. There is no single "best" sampler; you pick one to fit your objective.

The paper's theoretical section explains why the two metrics diverge. Statistical consistency of training on a subsample is one condition. Recovering features one to one requires stricter identifiability and optimization conditions. In other words, even if a subsample gives a good approximation of the target objective, that does not guarantee that different runs will arrive at the same set of features.

What to Do in Practice

The takeaway for teams training SAEs is fairly down-to-earth. If the goal is to quickly get a working SAE with good reconstruction, a budget of hundreds of millions of tokens is, judging by these results, often excessive. That makes experiments cheaper, especially when sweeping architectures and hyperparameters.

If you publish claims about specific features, such as "this latent encodes such-and-such concept," reconstruction metrics alone are not enough. You need an explicit reproducibility check: several independent runs and a matching of features between them. A low match rate at a small budget, like the same 2% at 5M tokens, is a warning that individual features may be artifacts of a particular training run.

Finally, choose the sampler to fit the goal. For reconstruction, start with random sampling; for interpretability as measured by AutoInterp, test diversity-based selection. The authors stress that with aligned objectives and budgets, SAE research can become cheaper and more reliable. One caveat: the figures apply to specific models and architectures, and transferring them to other scales requires separate verification.