OpenAI wraps up Parameter Golf: how coding agents are changing machine learning research

OpenAI has wrapped up Parameter Golf, an open machine learning competition that forced participants to find unconventional solutions within very tight constraints. Over eight weeks, the challenge drew more than 1,000 participants and over 2,000 submissions. The biggest surprise was how much AI agents changed the research process itself.

How the challenge worked

Parameter Golf was simple only on paper. Participants had to minimize held-out loss on a fixed FineWeb dataset while fitting the entire artifact, model weights and training code together, into 16 MB. There was one more constraint: training could not take longer than ten minutes on eight H100 accelerators. OpenAI chose this configuration deliberately, so the task would stay verifiable without reducing to brute-force search.

The organizers provided a baseline, the dataset and evaluation scripts, and accepted submissions through GitHub. As a result, the contest was open not only to researchers at large labs but also to independent developers who can experiment quickly and carefully build improvements on top of other people's ideas. OpenAI notes that the format turned out to be a good way to find strong engineers: it reveals not only knowledge of theory but also research taste, persistence and discipline.

What participants found

The strongest results came not from one magic idea but from many precise technical decisions. Some squeezed more quality out of known components by carefully tuning the optimizer, initialization and training schedule. Others focused on compression to fit the model into the strict size limit. Some entries sat at the edge of what was allowed, where a model improvement nearly blended into the evaluation strategy, so organizers had to check separately whether such tricks broke the spirit of the rules.

• Training tuning: participants combined improvements already found and reached even lower error without changing the underlying idea.
• Quantization: GPTQ-lite and full Hessian GPTQ entered the competition in force for the first time, as ways to compress weights more aggressively after training.
• Adaptation at evaluation time: some entries used test-time LoRA and similar approaches while staying within the formal rules.
• New data representations: unconventional tokenizers appeared, along with ways to handle letter case or the byte structure of text without loss.
• Architectural moves: participants tried partial attention variants, hash features for neighboring tokens, and even reusing layers as a recurrent mechanism.

OpenAI singled out the nonrecord track, a more experimental lane where technical boldness mattered more than absolute leaderboard position. It saw ideas such as state-space models combined with JEPA, Guided Attention, a byte-level H-Net, non-autoregressive text modeling and dynamic tokenization. The track was not decorative: half of the entries beat the naive baseline of 1.22 BPB, and the best result reached 1.12 BPB. That is an important signal that, even against strong transformer baselines, alternative approaches can still compete.

How AI agents affected the contest

The main difference between Parameter Golf and similar competitions in past years was the widespread use of coding agents. According to OpenAI, the overwhelming majority of participants mentioned working with agents. This sharply lowered the barrier to entry: it became easier to set up an environment, make sense of unfamiliar code, test a hypothesis quickly and put together a working submission without long manual drudgery. Infrastructure helped too: RunPod's sponsorship program gave participants $1 million in compute, so more people could experiment.

But speed came with noise. Many new submissions were not independent breakthroughs but small variations on already successful solutions. That is not a problem in itself, since good ideas should spread quickly. The problem is different: once a strong but invalid trick came to light, other agents began copying and scaling it, continuing down the wrong path. As a result, verification, attribution of contributions and correct scoring became noticeably harder than in contests before the era of agentic programming.

The flood of entries also changed the operational side of the contest. When hundreds of submissions arrived on some days, manual review stopped working. So OpenAI built an internal triage bot on top of Codex, which tracked new submissions and raised flags for manual checks. AI agents also became part of the community: one participant and their agent ran a live-updates bulletin on the leaderboard, and tools appeared around the contest that helped newcomers check their ideas against the rules.

What this means

Parameter Golf showed that AI-assisted research is already practice, not a pretty hypothesis. Agents speed up entry into ML, make experiments cheaper and widen the pool of participants, but they also change the mechanics of research competitions. If such formats are repeated, organizers will have to design not only the task but also a system for filtering, review and fair accounting of contributions in a world where code is increasingly written by more than one person.