DeepSeek V3.2 достигла 67% точности на тесте абстрактного мышления ARC-AGI

Researchers published on arXiv a method that allows the open model DeepSeek V3.2 to solve abstract reasoning test ARC-AGI-1 tasks with 67% accuracy on pass@2, using only agent architectures, operating in non-thinking-token mode and without specialized training on ARC data.

• DeepSeek V3.2 — open model, non-thinking mode
• Explorer-Definer Pipeline: 57.50% pass@2 for $0.25 per task
• Reflective Orchestrator: 67.25% pass@2 for $0.62 per task
• 52-point improvement from baseline single-shot result (15.50%)
• Finding: quality of variant generation matters more than their selection

How the two-stage agent works

Researchers divided the task into two stages: pattern discovery and program synthesis. Explorer-Definer Pipeline is two sequential agents. In the first stage, the model finds patterns in examples; in the second stage, it transforms patterns into executable transformation code. This division allows systematic problem decomposition instead of attempting to solve it from scratch.

Result: the pipeline achieved 57.50% pass@2 accuracy on a public set of 400 tasks, spending $0.25 per task. For comparison, the baseline single-shot (one example) yielded 15.50%.

Reflective Orchestrator: iterative re-exploration

Based on the pipeline, the authors built Reflective Orchestrator—a system that autonomously re-explores the task when a hypothesis fails on training examples. When the found transformation doesn't work, Orchestrator searches for new variants rather than simply iterating through already found ones. This increased accuracy to 67.25% pass@2, but cost rose to $0.62 per task—a trade-off between accuracy and budget.

Critical observation: analysis showed that the model is generation-bound, not selection-bound. Selecting the best answer from those proposed captures ~95% of potential, the remainder must come from expanding generation itself. Orchestrator confirmed this: unbiased pass@1 improved by 9.81 points thanks to new variants, not re-ranking of old ones.

Why this challenges frontier models

The authors bypassed two popular approaches: heavy test-time compute (evolutionary search, chain-of-thought) over GPT-5/Claude and benchmark-specific fine-tuning of small models. Instead, they showed: open DeepSeek V3.2 achieves 67% through agent-type architecture alone, without additional training. The model's thinking tool is a notable component: disabling it reduced results by 5.75 points, confirming hidden reasoning even in non-thinking mode.

What this means

The study proposes an alternative to scale: correct task division and iterative re-exploration yield greater gains than simply a larger model with more compute. For developers, this is a signal: a well-designed system can surpass scale through architectural engineering.

Frequently asked questions

How does ARC-AGI-1 differ from typical ability tests?

ARC-AGI-1 is 400 tasks on discovering hidden patterns. The model sees 3–5 examples (inputs-outputs) and must propose transformation logic for new inputs. This is closer to "learning from examples in a few steps" than to knowledge questions.

Why is this cheaper than fine-tuning?

Fine-tuning requires dataset preparation, training, validation on GPU. Here—only inference with agent calls: three or four model accesses per task. Everything happens at inference-time, without weight updates.

When will this appear in real products?

For now this is research on arXiv from July 6, 2026. Implementation in products depends on method generalizability beyond ARC-AGI—currently unclear.