Artificial Analysis launches AA-3D: Claude Sonnet 5.5 leads with 1194 Elo

Artificial Analysis has released AA-3D, a benchmark that tests whether AI models can build interactive 3D scenes that people actually want to explore. The launch leader is Claude Sonnet 5.5 with 1194 Elo. However, the confidence intervals of the top four models, all from Anthropic and OpenAI, overlap, so there is no single clear winner. Meta's Muse Spark 1.3, in seventh place, scored 1035, while the best open models, Kimi K3 (982) and MiMo-V2.6-Pro (977), take 11th and 12th place, more than 200 points behind.

How the test works

Each model works as an agent in an offline sandbox. It receives a text brief for a single scene, such as "Inside a mitochondrion," "A beating heart," "A cat clearing a table," or "Anatomy of a weightlifter." Every model also gets the same submission contract. The agent has access to Blender, Rapier physics, Python, a headless browser, and a library of CC0 assets. It writes code, renders the scene, looks at its own screenshots, and makes corrections. The limit is three hours or 250 turns. The output is a single index.html file built on three.js.

Scoring then relies on blind pairwise voting: a recruited panel picks the better of two scenes for the same brief without knowing who made them. The final ranking is calculated with Elo. The sample covers 21 models, 1008 scenes, and more than 10 thousand votes.

What sets the winners apart

The main finding: what matters is not how much a model works, but how it works. Output token count and number of turns do not correlate with Elo (Spearman 0.33 and −0.27, both not significant). Claude Sonnet 5.5 generates 569 thousand output tokens per scene, while GPT-6 Sol generates only 109 thousand, yet Sol ranks fifth.

• Early look at the result. The top 5 models first look at a screenshot of their scene on average at turn 10–13, and by turn 20 they have done so in at least 89% of scenes. Models ranked 14th through 20th wait until turn 15–40. The correlation between time of first look and Elo is 0.82. MiniMax-M3 looks at its scene 54 times, more than any other model, but it gets there by turn 20 in only 7% of cases and ranks 19th.
• Interface. Leaders have a median of 15–20 buttons and controls on the page, versus 6–8 for the bottom models (Pearson 0.90).
• Detail. The most detailed fifth of scenes wins 63% of votes, while the sparsest fifth wins 31%. But this is largely down to the stronger models: within a single model, added detail is worth about 1.5 points per standard deviation.
• Performance does not separate the models: the median frame takes less than 1.3 ms of GPU time for all of them.
• CC0 assets appear in three scenes out of four, but they do not affect wins once the opponent is taken into account.

The role of vision stands out separately. GLM-5.3 and Nemotron 3 Ultra cannot process images and built their scenes blind. GLM-5.3 lands 197 points below the trend, while GLM-5.3-Flash, which can see images, scores 149 points higher despite a lower Intelligence Index. General intelligence explains a lot: each index point is worth about 18 Elo (Pearson 0.92). But OpenAI models beat this trend: GPT-6 Luna by 141, GPT-6 Sol by 93, and GPT-6 Astra by 46.

Cost and speed

A scene costs between $0.20 and $32.08. GPT-6 Luna is the cheapest and Claude Fable 5.1 the most expensive. Luna still builds all 55 scenes and ranks eighth. Kimi K3 spends 68 times more per scene for a comparable Elo. Input tokens account for 69% of spending, because the agent resends the entire context on every turn. Among the top five, GPT-6 Sol is the cheapest at $3.46 per scene. Sonnet 5.5 costs 4.4 times as much, and Fable 5.1 costs 9.3 times as much.

Working styles differ too. GPT-6 Astra produces its first complete page after 6 minutes and finishes at minute 68. Sonnet 5.5 shows nothing until minute 42 and works for almost three hours, yet its run costs $14.93 versus $11.62 for Astra, even though it writes 6.2 times more tokens: low list prices help. The median GPT-6 Sol run takes 60 minutes, versus 115–151 for the other four leaders.

What to take away

If you build agents that generate interfaces, visualizations, or games, build early and frequent inspection of the agent's own render into the loop: it is the most robust quality signal in the data. Choose a model without access to images only deliberately. For bulk scene generation on a budget, GPT-6 Luna and GPT-6 Sol are the best fits: quality is close to the top group, and the price is an order of magnitude lower.

For open models, the gap remains large, so on 3D tasks going local still costs more in quality. Keep the limitations in mind as well: the tie among the top four means you should choose between them by price and run time, not by a row in the table.