DuoDiscover, built on coding agents, speeds up vLLM 1.92x and beats ArchAgent v2

The main problem with autonomous optimization search in real engineering is the cost of a single check. A full benchmark or serving trace takes hours, so only a handful of attempts get measured against the real thing. The authors of DuoDiscover propose a way around this limit: let the agents invent a cheap substitute for the real evaluation themselves, and make sure it does not lie. On two repository-level benchmarks and four sets of GPU and TPU kernels, the system outperforms human experts. On a CPU design task, it beats ArchAgent v2 on AlphaEvolve with a search time several times shorter.

A yardstick instead of a full run

DuoDiscover has three mechanisms. First, every worker agent in the fleet of coding agents simultaneously finds an optimization target and builds a fast, custom evaluator for it. The authors call it a yardstick. The agent then climbs the gradient of this yardstick, not of the real metric. The cost per step drops by orders of magnitude, so the search runs many iterations where it previously could afford five.

This approach has an obvious weakness. The search can exploit the gap between the proxy and the real objective: the yardstick keeps rising while real speed stays flat or drops. That is why the second mechanism is fidelity tests. They regularly check whether the ranking of candidates by the yardstick matches their ranking by the true evaluation. If the yardstick has "drifted," it is declared invalid.

The third mechanism concerns organization. The fleet works bottom-up, with no central decomposition of the task. A lightweight orchestrator launches and terminates agents, updates the briefing for running ones, and seeds newcomers with verified work from others. Nobody splits the problem into subtasks in advance; structure emerges from what the agents find.

What the measurements showed

The system was tested on two repository-level benchmarks and on four sets of GPU and TPU kernels, hundreds of kernels in total. Key figures from the paper:

• SWE-fficiency: average speedup twice that of human experts.
• SOL-Exec: first place on 121 of 235 kernels, top 3 on 212.
• vLLM: 1.92x speedup over the baseline code.
• SQLite: 1.38x speedup.
• CPU microarchitecture: evaluations of up to 28 hours, with a result better than both ArchAgent v2 and the best human design.
• Same CPU case: ArchAgent v2's 1.65 gain took 16.7 days versus more than 40 days.

vLLM and SQLite are especially telling. These are mature codebases that have been optimized for years, where the easy wins have usually been taken already. A near-2x speedup for vLLM means the agents found headroom that the development team had not closed. The chip case is even more interesting. When a single measurement takes more than a day, classical evolutionary search runs into its time budget. Here, the yardstick made it possible to finish in 16.7 days, and the result beat both the previous AlphaEvolve-based system and the human design.

Who can use this in practice

The idea applies well beyond the paper. Any team whose regression run, load test, or simulation takes hours hits the same wall. The practical takeaway: before scaling up the number of agents, invest in a fast proxy and in a test of its honesty. Without a ranking check, the agent will optimize a measurement artifact.

For infrastructure teams that work on inference and kernels, the paper offers a benchmark: an autonomous fleet can already compete with kernel experts and find gains in vLLM. For chip developers, something else matters more. Cutting the search cycle from 40+ days to 16.7 changes the economics of microarchitecture experiments.

The limitations are clear as well. The results were obtained by the authors on selected benchmarks, and there is no independent verification yet. The compute cost of the fleet is not disclosed in the announcement, so a fair comparison with humans or ArchAgent v2 is possible only on search time, not on cost. But the direction is clear: the bottleneck of autonomous research is becoming not idea generation but cheap, reliable verification, and DuoDiscover builds its entire architecture around that.