OpenAI: две настройки API утроили результат GPT-5.6 на ARC-AGI-3 — до 38,3%

OpenAI announced on July 30, 2026 that enabling two settings in its Responses API — retained reasoning and compaction — tripled the score of the GPT-5.6 Sol model on the ARC-AGI-3 benchmark: from 13.3% to 38.3% on the public task set. At the same time, output token spend per game dropped roughly sixfold, and with the new configuration the model surpassed Anthropic Opus 5.

What ARC-AGI-3 tests

ARC-AGI-3 is an interactive benchmark for abstract reasoning, where an AI agent works through unfamiliar game tasks turn by turn: it takes an action, observes the result, and plans the next move. Unlike static tests, memory between turns is critical here — the agent must accumulate an understanding of the game's rules as it progresses. It is precisely this memory that is behind the entire jump from 13.3% to 38.3% that OpenAI writes about.

What the two settings are

The two settings — retained reasoning and compaction — are the same mechanisms OpenAI uses in its ChatGPT and Codex products. Retained reasoning preserves the model's private chain of reasoning between separate context windows. Compaction summarizes accumulated context when it approaches the limit, instead of mechanically trimming the oldest material. Both parameters are available through the Responses API — OpenAI's interface for agentic scenarios.

• Model — GPT-5.6 Sol from OpenAI
• Official ARC-AGI-3 harness: 13.3% on the public set
• With retained reasoning and compaction via the Responses API: 38.3%
• Output tokens per game — roughly 6x fewer
• Publication — OpenAI blog, July 30, 2026

Why the score was lower

After each of the agent's turns, the official ARC-AGI-3 harness discarded its private reasoning, which kept the score stuck at 13.3%. GPT-5.6 Sol saw a log of past actions and short notes about them, but not the thinking that produced those moves — so at every step it was effectively figuring out the game from scratch again. When the history of moves exceeded the context window, the harness also deleted the oldest actions. In essence, the model remembered that it had "moved the purple block," but forgot why it had done so.

"Benchmarks never measure the model alone — they measure the technical scaffolding around it too,"

OpenAI's blog states.

How GPT-5.6 compared with Opus 5

With the settings enabled, GPT-5.6 Sol scored 38.3% and surpassed Anthropic Opus 5, which scored 30.2% on ARC-AGI-3. According to OpenAI, the gap did not arise from the model itself but from the test configuration: the official harness lacks provider-specific capabilities such as retained reasoning, which are available through the Responses API. The same technique produced a side effect — roughly a sixfold saving in output tokens per game.

What this means

A benchmark result depends not only on the model, but also on how the API is configured and how the test harness is designed. The very same GPT-5.6 Sol shows 13.3% or 38.3% depending on the configuration — and this changes how the comparison tables that AI labs use to measure themselves against each other should be read.

Frequently asked questions

What are retained reasoning and compaction?

Retained reasoning preserves the model's private chain of reasoning between context windows, while compaction summarizes old context instead of simply trimming it. OpenAI already uses both settings in ChatGPT and Codex.

How much did GPT-5.6 Sol's results improve?

On the public ARC-AGI-3 set, the score rose from 13.3% to 38.3% — roughly threefold. At the same time, output token spend per game fell by around sixfold.

Did GPT-5.6 Sol beat the Opus 5 model?

Yes. With the two settings, GPT-5.6 Sol scored 38.3% versus 30.2% for Anthropic Opus 5 on ARC-AGI-3, according to OpenAI.