LM2LM: Qwen3.8-27B Beats GPT-6-sol at Ideas for Improving LLMs and Is 33% Cheaper

The idea of neural networks improving neural networks has long been discussed as a path to recursive self-improvement. The LM2LM team has demonstrated a working version: they used RL to train a language model to generate effective proposals for improving LMs. On held-out tasks across eight development environments, both autoregressive and diffusion, Qwen3.8-27B beat GPT-6-sol. GPT-6-sol lost even with 300 real experiment records in its context, and LM2LM's data generation cost 33% less.

Why an honest reward for an idea is hard to build

The main obstacle is economic. To teach a model to propose good improvements, RL has to score tens of thousands of proposals. In the limit, each one requires a full training run of a model. At ten GPU-hours per run, the rewards alone would consume hundreds of thousands of GPU-hours. That budget is unrealistic for most labs, so they need a cheap substitute.

The substitute has its own problem. A surrogate score is reliable only as long as the policy generates proposals similar to those the surrogate was built on. Once optimization starts shifting the distribution of proposals, the policy finds the ones the surrogate overestimates. This is classic reward hacking: the model learns to please the evaluator rather than to improve real models.

A research world model and a reality check

LM2LM solves both problems with a combination of two elements. The first is a research world model (RWM): a frozen LLM that receives records of real experiments in its context. From them, it predicts how much gain a new proposal will deliver. The RWM serves as the reward model for RL and replaces thousands of full training runs with a single model call.

The second element is periodic verification. The current policy's proposals that the RWM scored highest are actually executed, and the measured results are added to the RWM's context. This way the evaluator sees exactly the region of idea space the policy has drifted into, and it recalibrates where the policy is most likely to fool it. The most suspect candidates, those that received high scores, are checked with a real run.

Key parameters of the work:

• policy: Qwen3.8-27B trained with RL;
• reward model: a frozen LLM (RWM) with real experiment records in its context;
• baseline: GPT-6-sol, given 300 real records in its context;
• environments: eight LM development environments, autoregressive and diffusion;
• evaluation: real execution of proposals on held-out tasks;
• savings: data generation costs 33% less;
• generalization: transfer to a model family not seen during training.

The comparison conditions deserve a separate note. GPT-6-sol got a strong hint in the form of 300 real records, yet the smaller model with a trained skill still came out ahead. It appears that accumulated experience is better passed on through policy training than through a long context in a more general model.

Where this is useful in practice

For teams that run many model-training experiments, the most valuable point is that logs of past runs become an asset. Records of which changes to hyperparameters, data, or architecture worked can serve as context for the RWM and as a source of training signal for the idea generator. Expensive runs are spent only on verifying the most promising candidates.

There is also a cautionary takeaway. The scheme works only with discipline: without regular checks against real runs, the surrogate will start overestimating ideas, and the policy will follow it. The budget for real experiments has to be built into the pipeline as a permanent line item. Transfer to an unfamiliar model family suggests that the accumulated experience is not tied to a single architecture, but teams will still have to verify it on their own stacks.