SubQ Snap 1.2 Rises from 68 to 73 Points on Decision Index: It Says 'I Don't Know' More Often
The Decision Index tests whether a model is fit for decision-making: whether it makes up answers and whether it changes its choice when the question is reworded. SubQ Snap 1.2 scored 73 points against 68 for version 1.1, drawing level with Drex Max v1.1 and GLiDE v1.2. Accuracy did not rise (21% and 9%). Instead, the model refuses to answer more often when it is unsure, so the share of answers without fabrications rose from 85% to 89%. For agents where errors are costly, that is a plus.
AI-processed from X: @ArtificialAnlys; edited by Hamidun News
Imagine an agent that decides for itself which action to take: approve a payment, reply to a customer, close a ticket. Average accuracy tells you little here. What matters more is that the model does not make up an answer when it doesn't know, and does not change its decision if the same options are simply reworded. Independent lab Artificial Analysis evaluates exactly these properties in its Decision Index. In version v2.1, Subquadratic's SubQ Snap 1.2 scored 73 points against 68 for its predecessor SubQ Snap 1.1. That is a gain of 5 points, and it draws the model level with the leaders of the ranking. But the improvement came from an unexpected place.
What the ranking showed
Two competitors, Drex Max v1.1 and GLiDE v1.2, also scored 73 points, and the gap with SubQ Snap 1.2 is within 0.1 points. In effect, three models are competing for first place, and you cannot choose between them based on the ranking score alone.
The price stayed the same: $0.04 per million input tokens. The context window is 384 thousand tokens. The improvement came without a price increase, which matters if you are counting the cost of processing large volumes of text.
The two subtests that grew the most were AA-Paperclip and AA-Symmetry, with 8 points each.
Where the growth came from
AA-Paperclip tests work with documents. It measures the share of documents the model answered completely correctly, without a single error. For SubQ Snap 1.2, this rose from 46% to 62%. Drex Max v1.1 scores 63% on this measure and GLiDE v1.2 scores 62%, so here all three models are nearly equal.
AA-Symmetry tests stability: does the model keep the same decision and the same confidence when the question changes in a way that should not affect the answer? For example, you ask an agent to choose one of three action options, then restate those options in different words. A reliable model will choose the same thing. In Decision Index v2.1, this test was updated to version 1.1: the answer options now sit inside the description of the situation and are reworded while keeping the same meaning. The task became harder, yet SubQ Snap 1.2 still improved its result by 8 points.
Caution instead of accuracy
The two other tests, AA-Omniscience-Decision and HLE-MultiChoice, are the most interesting. The number of correct answers did not increase: accuracy stayed at 21% and 9%. However, the share of answers without fabrications rose, meaning cases where the model did not give a confidently wrong answer: from 85% to 89% in the first test and from 46% to 55% in the second.
The reason is refusals. On AA-Omniscience-Decision, the model now attempts to answer 29% of questions, down from 33%. It more often admits it doesn't know, and less often offers an incorrect guess. The ranking counts this as an improvement, because a fabricated answer is penalized more heavily than an honest refusal.
What practitioners should do
First, look at what the gain is made of. Five points do not mean the model has become smarter: a noticeable part of the improvement comes from caution. For scenarios where errors are costly (finance, medicine, legal documents, automated agent actions), this is good. If, however, you need some answer to every request, excess caution will get in the way.
Second, measure the refusal rate on your own data. If the model answers 29% of questions instead of 33%, your product's behavior will change. The interface must be able to handle "I don't know": passing the question to a human or to another model.
Third, when scores are equal, compare price, speed and context length. At $0.04 per million input tokens and a 384-thousand-token window, SubQ Snap 1.2 looks like a reasonable choice for bulk processing of long documents.
Limitations: these are benchmarks from a single independent analyst, and the stability test was recently updated, so the numbers cannot be compared directly with past versions of the index. Results on a real task may differ. Before switching to the new version, run your own set of examples.
- Nov 21, 2026Meta Is Reportedly Preparing the $300 Muse Charm AI Tamagotchi With a Year of Verizon 5G
- Nov 21, 2026Dreamteam Raises $53M for a CRM That Fills Itself from Email and Calls
- Nov 21, 202622 Claude Agents Spent Six Weeks on Black Hole Problems — and Solved Two
- Nov 21, 2026Google built a closed copy of the internet with fake banks to train Chrome's AI agents
Key Questions Answered
What the ranking showed?
Two competitors, Drex Max v1.1 and GLiDE v1.2, also scored 73 points, and the gap with SubQ Snap 1.2 is within 0.1 points. In effect, three models are competing for first place, and you cannot choose between them based on the ranking score alone. The price stayed the same: $0.04 per million input tokens. The context window is 384 thousand tokens. The improvement came without a price increase, which matters if you are counting the cost of processing large volumes of text.
Where the growth came from?
AA-Paperclip tests work with documents. It measures the share of documents the model answered completely correctly, without a single error. For SubQ Snap 1.2, this rose from 46% to 62%. Drex Max v1.1 scores 63% on this measure and GLiDE v1.2 scores 62%, so here all three models are nearly equal. AA-Symmetry tests stability: does the model keep the same decision and the same confidence when the question changes in a way that should not affect the answer?
What are the key details on Caution instead of accuracy?
The two other tests, AA-Omniscience-Decision and HLE-MultiChoice, are the most interesting. The number of correct answers did not increase: accuracy stayed at 21% and 9%. However, the share of answers without fabrications rose, meaning cases where the model did not give a confidently wrong answer: from 85% to 89% in the first test and from 46% to 55% in the second. The reason is refusals. On AA-Omniscience-Decision, the model now attempts to answer 29% of questions, down from 33%.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.