OpenAI: Two API Settings Tripled GPT-5.6's ARC-AGI-3 Score — to 38.3%

On July 30, 2026, OpenAI reported that turning on two settings in its Responses API, retained reasoning and compaction, tripled the score of GPT-5.6 Sol on the ARC-AGI-3 benchmark: from 13.3% to 38.3% on the public set of tasks. Output token use per game fell roughly sixfold, and with the new configuration the model beat Anthropic's Opus 5.

What ARC-AGI-3 tests

ARC-AGI-3 is an interactive benchmark for abstract reasoning. An AI agent works through unfamiliar game tasks turn by turn: it takes an action, sees the result, and plans the next move. Unlike static tests, memory between turns is critical here, because the agent has to build up an understanding of the game's rules as it plays. That memory is behind the entire jump from 13.3% to 38.3% that OpenAI describes.

What the two settings do

The two settings, retained reasoning and compaction, are the same mechanisms OpenAI uses in its ChatGPT and Codex products. Retained reasoning preserves the model's private chain of reasoning across separate context windows. Compaction summarizes the accumulated context as it approaches the limit, instead of mechanically trimming the oldest material. Both parameters are available through the Responses API, OpenAI's interface for agentic workflows.

• Model: GPT-5.6 Sol from OpenAI
• Official ARC-AGI-3 harness: 13.3% on the public set
• With retained reasoning and compaction via the Responses API: 38.3%
• Output tokens per game: roughly 6 times fewer
• Publication: OpenAI blog, July 30, 2026

Why the score was low

After each agent move, the official ARC-AGI-3 harness threw away the agent's private reasoning, which kept the score stuck at 13.3%. GPT-5.6 Sol saw a log of its past actions and short notes on them, but not the thinking that produced those moves, so at every step it effectively had to figure out the game anew. When the move history exceeded the context window, the harness also deleted the oldest actions. In effect, the model remembered that it had "moved the purple block" but forgot why it did so.

"Benchmarks never measure only the model — they also measure the technical scaffolding around it,"

OpenAI says in its blog.

How GPT-5.6 stacks up against Opus 5

With the settings enabled, GPT-5.6 Sol scored 38.3%, ahead of Anthropic's Opus 5 at 30.2% on ARC-AGI-3. According to OpenAI, the gap came not from the models themselves but from the test configuration: the official harness lacks provider-specific features such as retained reasoning, which are available through the Responses API. The same technique also had a side effect: output tokens per game dropped roughly sixfold.

What this means

A benchmark result depends not only on the model but also on how the API is configured and how the test harness is designed. The same GPT-5.6 Sol scores either 13.3% or 38.3% depending on configuration, and that changes how comparison tables should be read when AI labs use them to measure themselves against each other.

Frequently asked questions

What are retained reasoning and compaction?

Retained reasoning preserves the model's private chain of reasoning across context windows, while compaction summarizes old context instead of simply trimming it. OpenAI already uses both settings in ChatGPT and Codex.

How much did GPT-5.6 Sol's results improve?

On the public ARC-AGI-3 set, the score rose from 13.3% to 38.3%, roughly a threefold increase. Output token use per game fell about sixfold.

Did GPT-5.6 Sol beat Opus 5?

Yes. With the two settings, GPT-5.6 Sol scored 38.3% against 30.2% for Anthropic's Opus 5 on ARC-AGI-3, according to OpenAI.