GPT-6 Astra changed 0.016% of Qwen's weights and raised accuracy to 48.1%

When a small language model gets stuck in loops, performs calculations poorly, or ignores instructions, the usual prescription is familiar: collect data, run fine-tuning, and check whether anything else has broken. MIT researchers demonstrated another approach. GPT-6 Astra was given access to the model's weights, found the region associated with the fault, and rewrote a tiny fraction of the parameters. In an experiment with Qwen3.5-0.8B, changing just 0.016% of the weights raised accuracy on MATH-500 from 7.9% to 48.1%.

This is neither a magic button nor a replacement for training. But for teams maintaining their own models, the idea is highly practical: instead of running an expensive pass over the entire dataset, they could try to identify a specific internal defect and repair it precisely.

How one model programs another

A neural network's weights are enormous tables of numbers. A person generally cannot inspect them and determine what the network was trained to do or why it behaves in a particular way. In the experiments, Astra handled some tasks of this kind: it identified the game for which a small network had been trained, recognized the type of images produced by a generator, and matched small language models to their training languages. It did all this without the original training data and, in some cases, without even running the model itself.

The clearest example involves simple computer games. Researchers gave Astra the rules and code, after which it directly created the weights for game-playing networks. In Breakout, such a network scored an average of 517 points, compared with 25 for a baseline model trained conventionally for more than 100 million rounds. However, in two other games, the models trained for the longest time were still stronger. This is therefore not yet a victory over conventional training, but a new way to work with small, tightly constrained tasks.

In an experiment with Go on a 9×9 board, Astra controlled a network with roughly 1.5 million parameters. It was given 50 iterations: in each one, the network played up to 128 training games, while Astra studied the outcome and adjusted the weights. The resulting model outperformed a similarly sized network trained on roughly 20 million games.

What happened with Qwen

Qwen3.5-0.8B had a familiar problem: the model could enter a loop and repeat its reasoning without ever reaching an answer. Astra received a coding environment, training tasks, and 50 attempts to produce a fix. The result was then evaluated on different examples that Astra had not seen during the adjustment process.

After 0.016% of the parameters were changed, accuracy on MATH-500 rose from 7.9% to 48.1%, while the instruction-following score increased from 21% to 62%. The share of responses that hit the maximum length fell from 93% to roughly 33%. That is still a large proportion, so it is too early to consider the problem solved.

The negative result is also revealing. On Gemma 3 270M, which barely suffered from such loops, Astra could not deliver a measurable improvement. The authors suggest that Qwen already possessed the necessary mathematical capabilities, but an internal fault prevented it from using them. Astra removed this "brake" rather than inserting a new skill into the model.

What developers should do

For now, Weight-Space Programming should be treated as a research tool. It could be useful for finding defects, conducting security audits, and precisely correcting the behavior of small models. But it is too early to transfer the results to large production LLMs: the experiments are limited, there is no universal test yet, and some tasks required running the model and using feedback from the results.

The practical takeaway is straightforward. If a team maintains a small specialized model, it should watch this field and retain detailed behavioral tests: those tests are what make it possible to determine whether a targeted repair is better than full fine-tuning. But the weights of a production model must not be changed without independent evaluation, regression tests, and a rollback option. Targeted repair may look cheaper than training, but the cost of a mistake does not disappear.