Claude fixed 165 weights in a medical model in an hour, raising recall from 76% to 90%

Two weeks ago, MIT published the paper Sparks of Weight-Space Programming: an LLM receives a file containing another model's weights, figures out how it works, and edits it without training. The headline result was that a small Qwen model jumped from 7.9% to 48.1% correct answers on school-level math. A product manager working in MedTech R&D decided to test the idea on a medical model. It worked, but with caveats more important than the success itself.

What the experiment involved

The experiment used OpenSpliceAI, an open version of SpliceAI. This network examines a DNA segment and predicts where a gene is "cut" during protein assembly. To assess a mutation, the model is run twice, on the original sequence and the altered one, and the difference between the outputs is measured. This score is used in ClinGen recommendations, so conclusions from real genetic laboratories depend on it. The model is small, with 699,000 parameters, and runs on a laptop without a GPU. Unlike the original, it has an open license.

Claude Opus 5.5 was tasked with improving the distinction between dangerous and benign mutations from ClinVar. It was not allowed to train the model or automatically search through values. Testing used chromosomes Claude had not seen, while also checking that the model's core performance did not deteriorate.

What Claude found

The diagnosis, rather than the edit, is the most interesting part. The model identifies genuine splice sites with a probability of about 99%. A dangerous mutation substantially reduces confidence inside the network, but this is barely visible at the output: at such values, the probability curve is flat. The model "knows" that the site is damaged, but the final score does not show it.

The fix proved simple. Claude multiplied the final layer's logits by 0.35, effectively increasing the softmax temperature, and doubled the signal from the DNA region near the splice site. Just two edits, changing 165 weights. On the hidden test, AUPRC rose from 0.939 to 0.975, while the share of dangerous mutations detected at the same number of false positives increased from 76% to 90%.

The catch: gradients perform just as well

The author then honestly tested the alternatives. The temperature adjustment alone accounted for 86% of the gain, and a simple search would have found the same value. Conventional fine-tuning of the same weights produced an AUPRC of 0.970–0.971, within the margin of error of Claude's result. An ensemble of five copies scored 0.953, so Claude won there. The same was true in MIT's experiment: if reasoning mode is disabled on the original Qwen model, it scores about 46% by itself. The edit removes looping rather than teaching mathematics.

The metric is a draw. In terms of understanding, Claude wins: a gradient returns new weights but says nothing about the reasons, while the LLM explained within an hour what was breaking the score and where it could be fixed.

Limitations

• One experiment, one model, one Claude run.
• The probability scale shifted, making the clinical thresholds invalid: at a threshold of 0.8, the model detects 15% of dangerous mutations instead of 49%. Recalibration is required.
• The temperature trick is specific to this setup: the score here is the difference between two probabilities; it would provide no benefit for a conventional classifier.
• Some ClinVar labels were themselves based on SpliceAI.
• The model acquired no new capability: mutations it could not recognize are still missed.

Practical implications

Do not expect an LLM to "rewire" your weights better than a gradient. But it can be useful as a debugging tool: ask it to inspect the network, find where the signal is being lost, and propose a hypothesis. Then test that hypothesis with a cheap parameter search and fine-tuning. If a simple adjustment delivers 86% of the effect, it may be all you need, while the agent's value lies in explaining why. The code and tables are available on GitHub, and the experiment can be reproduced on a CPU.