OpenAI unveils BioBasicsBench: GPT-6 Astra scores 66.7% on wet-lab data analysis

If you are building an AI agent for science, the key question is not whether it can summarize papers. What matters is whether it can look at a blurry photo of a gel or a plate of colonies and honestly say: "no conclusion can be drawn from this." That is exactly what OpenAI's new benchmark, BioBasicsBench, built together with the biotech company Ligo, tests.

What is inside the benchmark

The set contains 100 tasks across five areas of routine laboratory work: protein methods, imaging and cytometry, quantitative assays, nucleic acid work, and molecular cloning. The data is raw: gels and blots, plate photos, microscopy stacks, FASTQ files, plate reader exports, even videos of bench work.

The tasks were written by scientists using data from their own experiments. Each one was checked by an internal Ligo reviewer, an external PhD-level specialist, and then by the team once more. Answers are graded against an expert rubric: points are awarded for individual criteria, so a correct conclusion with poor reasoning is distinguished from a fully correct answer. Some criteria check whether the agent recognized that the data was insufficient.

Seven models were evaluated across 37 reasoning settings, with five attempts per task, for a total of 18,500 results.

Results

At the xhigh setting, GPT-6 Astra scored 66.7% (confidence interval 60.3–73.1%), GPT-6.1 Sol 62.0%, GPT-5.6 Sol 43.8%, GPT-6 Luna 38.3%, and GPT-5.2 31.2%. Astra also uses fewer tokens, about 4,700 per task versus 9,100 for Sol. According to OpenAI, the key factor is stronger native multimodality: Astra sees details in photos better.

The examples are vivid. In a cloning task, Sol saw an empty control plate and accepted it as a good control check. Astra read the label "Kan50" on the rim, meaning kanamycin instead of ampicillin, and rejected the control. In another task, Sol looked for a biological explanation for disappearing cells, while Astra realized it was a z-stack, that is, imaging at different depths. In a third, Sol declared cells dead based on a dye signal, even though a second dye overlapped in spectrum; Astra requested additional controls.

Where models still break down

OpenAI candidly shows Astra's failures too. It correctly separated the colony groups on the gel but mixed up which was which, and chose colonies 4, 6, 7, 8 instead of the reference 5, 9, 10. The familiar assumption that "a larger PCR product means successful cloning" outweighed the control lanes. In another case, it generalized that vacuoles are darker than the cell, ignoring lighter vacuoles in the same field, even though it had itself noted that contrast depends on focus. And when counting colonies, it overestimated the result by roughly 12%, which directly inflates the estimate of transformation efficiency.

What to do about it

For teams building scientific agents, the takeaway is practical. First, evaluate not only the final answer but also the ability to say "the data is insufficient": this is a separate rubric criterion. Second, do not trust an agent to automatically select samples for the next step, such as choosing colonies for plasmid purification: the cost of an error cascades. Third, object counting in images is better verified with classical tools or by a human.

The limitations should also be considered: the benchmark is small at 100 tasks, the intervals are fairly wide, and the evaluation was run by OpenAI itself on its own models, with no comparison to models from other labs. Nevertheless, the format itself, based on imperfect real-world data with traps such as a wrong label, is closer to reality than curated images from papers. For product managers in biotech, it is a good template for their own internal agent evaluations.