Thinking Machines' Inkling 2 Small scores 81.7% on ARC-AGI-2 at $0.27 per task

Open models typically lose to closed ones precisely where the goal is not to recall something memorized, but to work out a new problem. That is why the latest result from Thinking Machines' Inkling 2 Small deserves attention: on the ARC-AGI benchmark, it outperformed every open model evaluated by ARC Prize.

What exactly the numbers show

ARC Prize published the results with Verified status, meaning the organizers ran the evaluation themselves rather than relying on the model's developers. On ARC-AGI-2, Inkling 2 Small scored 81.7% at a cost of $0.27 per task. On the easier ARC-AGI-1, it scored 96.5% at $0.08 per task.

According to ARC Prize, the model outperforms every open model it has evaluated on both the first and second versions of the test. Thinking Machines plans to release the full weights next week.

Why ARC-AGI is unlike conventional tests

Most benchmarks can be partially learned: similar tasks appear in one form or another in the training data. ARC-AGI works differently. A model is shown several pairs of "before and after images" and must infer the hidden rule, then apply it to a new example. It is like deducing the logic of an unfamiliar board game from two or three moves.

A high score here is therefore treated as evidence that a model can generalize rather than merely remember well. ARC-AGI-2 is also considerably harder than the first version, as the gap between 96.5% and 81.7% demonstrates.

Cost matters separately. The ARC Prize report lists the cost per task, which provides a fair basis for comparison: in practice, a model that solves slightly more problems but costs ten times as much may still be the worse option. Here, $0.27 for a difficult task looks very reasonable.

What this means in practice

If you are building agents or services in which a model must reason over unfamiliar inputs, you may soon have an open alternative to closed APIs. You can deploy it on your own infrastructure, fine-tune it and avoid dependence on third-party limits and pricing policies. This is particularly compelling for companies with strict data requirements.

What you can do right now:

• Prepare a small set of your own tasks for comparing the model with your current one. A benchmark cannot replace evaluation on your own data.
• Estimate what inference would cost at your workloads if the weights are indeed released. The cost per task in the ARC Prize report was measured under its own conditions, and your figure may differ.
• Watch the license. It will determine whether the model can be used in a commercial product.

Limitations

For now, this is a single post containing benchmark figures. The weights have not been published, so no one can independently verify either the result or the evaluation setup. We also do not know how the model will perform with long contexts, code generation or tool use. ARC-AGI measures one facet of intelligence, and a strong result there does not guarantee the same level of performance in your product.

Even so, the signal is clear: the gap between open and closed models on generalization tasks continues to narrow. If the weights arrive on schedule, developers will be able to test that claim on their own tasks next week.