GPT-6 Astra edits 0.016% of Qwen3.5-0.8B's weights and lifts its MATH-500 accuracy from 7.9% to 48.1%

The Weight-Space Programming paper, reported by the Telegram channel Neurogen with reference to MIT CSAIL, shows something unusual: the frontier model GPT-6 Astra can directly read and rewrite the weights of other neural networks. It uses no gradients and no fine-tuning, only code in a sandbox. The most striking example is the small Qwen3.5-0.8B (about 800M parameters). Astra changed just 131,072 of its weights, or 0.016%, and the model's accuracy on MATH-500 (a set of 500 competition-level math problems) rose from 7.9% to 48.1%.

How Astra cured Qwen3.5-0.8B's looping

The researchers asked Astra to improve Qwen3.5-0.8B at math. The model ran Qwen in Python and observed its behavior. It found that the small network constantly gets stuck in loops while reasoning: 85 of the first 96 answers hit the 32,768-token limit and were cut off without producing a result.

Astra then traced the problem to the MLP blocks of layers 15 and 19 and subtracted an unwanted direction from the weights. In total, 131,072 of 800M parameters were changed, which is 0.016%. This took 50 forward passes on training problems and a manual weight edit in the sandbox.

Results after the edit:

• MATH-500: from 7.9% to 48.1%;
• GSM8K (grade-school math word problems): from 13.8% to 67.1%;
• IFEval (an instruction-following benchmark): from 21% to 62.3%;
• share of answers that hit the token limit: from 93% to 33%.

The IFEval gain has a simple explanation: the loops were not a math problem but a general behavioral defect, so the model also got better at following instructions.

What else Astra can do in weight space

The authors demonstrated other tasks unrelated to math:

• from raw weights alone, and without a list of options, Astra identified which game a network was trained on, and got all 4 cases right;
• for small language models with 7.2M parameters, it named the language and found all the copying heads (attention heads that copy fragments of the context); the result matched direct measurements;
• in the Kinetix environment, it fixed a car that could not climb a hill: an edit to 3.5% of the weights let it pass all 1024 trials, where before it passed none;
• it wrote weights from scratch for Breakout, Seaquest and Asterix; in Breakout it scored 517 points, while PPO after 500M training steps scored 25.3;
• it built a 1.5M-parameter Go network with a single 336-line code run.

Where the hype ends

The author of the channel's write-up adds caveats. Astra did not teach Qwen math: it removed a bug that kept the model from finishing its solutions. On Gemma 3 270M, which does not loop but simply counts poorly, the approach did not work at all. A weight edit removes an obstacle but does not add a capability.

Technically, this is close to a steering vector (a shift of activations in a chosen direction), and people have edited weights by hand before: ROME and abliteration come to mind. The novelty lies elsewhere. Previously a human did this with specialized tools; here the model itself explored another network, found the problem and fixed it.

The result can be verified. The checkpoint is on Hugging Face, and there is a demo where the base Qwen loops while the fixed version answers. The researchers also rebuilt the weights with hash verification and re-ran the evaluation: MATH-500 came out at 51.0%, slightly above the original result. Logs of all experiments are published on GitHub.

The takeaway for practitioners: diagnosing a model and surgically editing its weights could become a job for an agent, especially when a small model has a specific behavioral defect such as looping. But there is no reason to expect that editing 0.016% of the parameters will turn a weak model into a strong one: it cures a specific breakage and does not add knowledge.