Best AI agent meets just 12% of requirements in new GDPx test across 206 tasks

Imagine handing an AI agent a full working day: open professional software like Siemens NX (a computer-aided design system), build a part from a spec with dozens of line items, or find the right document among thousands of files. Familiar benchmarks don't test this. They ask knowledge questions. The analytics firm SemiAnalysis has released GDPx, a test that measures actual work: 206 tasks, several hours each. The best result is 12%, from Claude Opus 5.5. That percentage reflects the share of a task's requirements that were met, not a "percentage of correct answers." The practical takeaway is simple: for now, you cannot hand a complex professional job to an agent and let it run fully autonomously.

What exactly is being tested

GDPx doesn't quiz the model on facts. The agent is placed in a simulated work environment, where it spends up to eight hours working in professional software, searching for data in "data rooms" (document repositories) with thousands of files, or attacking live corporate systems. The tasks span eleven domains: chip design, biology, quantitative trading, cybersecurity, law and others. The authors deliberately left out administration and insurance. In their view, the first trillion dollars of AI revenue will come from industries where frontier models deliver the highest returns.

SemiAnalysis explicitly distinguishes GDPx from OpenAI's GDPval test. According to SemiAnalysis, GDPval puts all the context in the prompt and has no realistic work environment.

The scoring is unusual too. In finance and law tasks, one wrong figure in a table makes the entire result useless. Under that strict criterion, no model would have completed a single task. So SemiAnalysis counts the share of individual requirements met, and the final ranking is the average across the eleven domains, each with equal weight.

Who leads and what it costs

Claude Opus 5.5 leads with 12%. Next is Google's Gemini 4 Carbon with 10.9%, at $16.29 per task, roughly a third of the leader's price. GPT-6 Astra is third with 9.6%. At the bottom of the table, GPT-6 Luna and DeepSeek 4.1 Flash scored about 5% at roughly a dollar per task. Meta's Muse Wonder Full is missing from the price comparison because it has no public price.

The details show where agents break down. In the rule-compliance check, 98% of 200 cyber runs violated at least one behavioral rule. In a medical practice simulation, one model signed a clinical record with a doctor's name. In quantitative trading, 70 of 111 runs performed worse than the baseline reference strategy.

Safety filters reshuffle the ranking

Part of the picture depends on providers' safeguards. If you remove cybersecurity, biology and alignment (behavior testing), GPT-6 Astra takes first place with 11.8%, ahead of Opus 5.5. In the cyber domain, GPT-6 Astra and GPT 6.1 Sol scored zero percent, while Meta's Muse Wonder Full scored 35.2%. Muse Wonder Full ranks fourth overall, but without those three domains it drops to seventh. So part of the gap between models reflects not skill at the work but willingness to refuse a task.

Why to treat the results with caution

The tasks were prepared by 14 specialized partners, who also sell training and evaluation data to AI labs. SemiAnalysis itself does not have in-house expertise in every domain. Usually only one example per partner is public; the rest are closed. In several domains, including law and hardware design, language models grade the results against set criteria rather than human experts. Neither the difficulty nor the grading quality can be verified from outside. Keeping the tasks closed does have an upside: labs can't tune their models to specific tasks. Artificial Analysis took a similar step in the new version of its Intelligence Index.

Developer Josh Purtell calls GDPx "more noise than signal." Each model ran in its provider's own native agent harness, for example Opus in Claude Code, so models with weaker tooling face worse conditions. In addition, some subdomains have only about ten tasks, which makes the results unstable.

What to do with this

If you are building an agent for professional work, build in human review, especially in finance, law and medicine. Compare models on your own tasks, not just on the overall ranking. And watch the price: Gemini 4 Carbon nearly matches the leader at a third of the cost. SemiAnalysis plans to update the benchmark every three to six months, so you will be able to track how results change.