The best AI model completes just 12% of work tasks in the new GDPx benchmark

Today's best model completes 12% of real professional tasks. The figure looks like a typo, especially if your repository already contains a project written by AI. But that is exactly the result reported by SemiAnalysis's new GDPx benchmark.

What the benchmark measures

SemiAnalysis is known for its analysis of the semiconductor market and AI economics. On November 2, the company published GDPx. Its authors—Max Kan, Andrew Megalaa and company founder Dylan Patel—conceived it as the toughest test of agent capabilities. It tests neither factual knowledge nor the ability to solve competition problems, but performance on assignments businesses are willing to pay for.

The authors' choice is telling: the company views AI through a financial lens, looking at the cost of chips, data centers and tokens. Its central question is not "How smart is the model?" but "Does it pay for itself?" Hence the focus on paid tasks.

Anthropic's Claude Opus 5.5 delivered the best overall result. That result was the same 12%. The other models scored lower.

An important caveat: the excerpt available to us contains no methodological details—the mix of tasks, the evaluation process or the results for other models. Before citing the figure in a presentation, read the original SemiAnalysis publication.

Why agents struggle

A real work assignment involves a sequence of steps rather than a single answer: understand the context, choose tools, do the work, check it and make corrections. Simple arithmetic illustrates the risk. If an agent succeeds at each step 90% of the time, it will successfully complete a ten-step sequence roughly 35% of the time. This is an illustration, not GDPx data, but the mechanism is the same: errors accumulate, and in commercial work an "almost right" result is often worth zero.

This explains the gap between how capable models feel and what the numbers show. Ask a model to write a function or sketch out a script, and it often gets it right. Hand it an entire assignment—with an incomplete brief, someone else's code and a requirement to get everything working—and the picture changes. An AI-written project that "seems to work" and a task completed well enough for a client to pay are different things.

What to do about it

For developers: do not judge an agent by its demo. Collect 20–30 real tasks of your own with clear acceptance criteria and run the model on them. That will be your personal mini-GDPx. Track the share of tasks completed without human corrections: that metric is more honest than any average score.

For product managers: keep a human in the loop. If a feature promises fully autonomous operation, its actual scope of usefulness could be several times smaller than the marketing slides suggest. Factor the cost of reviewing and refining the output into your economics.

For founders: 12% is no reason to shut down agent projects. It is a reason to choose narrow use cases with short sequences of steps and inexpensive verification, and to stop selling an "employee replacement." A benchmark with such a low performance ceiling also has a useful property: it is not saturated. There is room to improve, and it will make progress in subsequent models visible.

Limitations

One benchmark does not capture all of reality. Its authors defined what counts as a "real professional task," and a different set of tasks will produce different numbers. SemiAnalysis is also an analytics company, not an independent laboratory, so it is worth examining how reproducible the results are and whether the task set is openly available. Still, the signal is clear: there is a substantial gap between "writes code" and "gets the job done," and you should measure it on your own tasks.