Researchers Find a 0.4–1.1% Window in Pythia Pretraining Where the Readout Reorganizes

The common assumption is that a model has acquired a skill at the moment it starts producing the correct token. A new paper from Hugging Face Daily Papers shows that this logic introduces a shift. Hidden states can carry task information earlier than the model's output expresses it. Using Pythia as a case study, the authors found a narrow window, 0.4–1.1% of the way through pretraining, where the unembedding matrix (the readout) reorganizes and WordNet categories show up in its rows.

The output learns too

Token prediction passes through the unembedding matrix, which maps a hidden state to logits over the vocabulary. This matrix trains at the same time as everything else. A behavioral plot showing accuracy rising with step N therefore mixes two processes: what the model has already encoded internally, and what its readout has learned to express externally.

The authors tested this directly. Linear probes on hidden states in early checkpoints already decode task information, even though the model's own outputs do not yet show it. The result is a gap between knowledge and its expression, and that gap distorts the dating of many "transitions" in a model's development.

Parameter-trajectory crosscoding

To track the readout, the researchers propose parameter-trajectory crosscoding. The method trains a single sparse dictionary on the readout rows for each token at each checkpoint. Classic crosscoders work with activations and require running data through the model. Here, only checkpoint weights are needed, which considerably simplifies the analysis of open models with a saved training history.

Key findings:

• the readout reorganization window in Pythia: 0.4–1.1% of pretraining;
• within this window, WordNet categories become decodable from the readout rows;
• by the end of the window, probes on hidden states already read information that is absent from the outputs;
• the analysis needs only checkpoint weights, no activations;
• with a higher readout learning rate, the final loss barely changes.

A controlled learning-rate experiment

Correlation is not enough, so the authors ran an intervention. They trained small models from scratch that differed only in the readout learning rate. If the gap is determined by how fast the output layer learns, it should move with that rate.

That is what happened. With a higher readout learning rate, the readout forms earlier and the gap between hidden state and output closes faster. The final loss barely suffers. So the readout learning rate affects when a capability becomes visible from the outside, but not where the model ends up.

How to use the result in practice

The finding applies to the early phase of training, where many "sudden" transitions are dated. If you build capability-emergence curves, a single output-based metric is not enough. Put linear probes on hidden states alongside it and compare the dates: if the probe leads the output, you are most likely measuring readout maturation, not the emergence of the skill.

For those training their own models, this is also a reason to look at a separate learning rate for the unembedding. The work was tested on small models and on Pythia, so conclusions should be carried over to large modern architectures with caution. But for open checkpoints the tool is already usable, since everyone has the weights.