Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseInsight · On RadarX: @ArtificialAnlys→ original

SubQ Snap 1.2 Rises from 68 to 73 Points on Decision Index: It Says 'I Don't Know' More Often

The Decision Index tests whether a model is fit for decision-making: whether it makes up answers and whether it changes its choice when the question is reworded. SubQ Snap 1.2 scored 73 points against 68 for version 1.1, drawing level with Drex Max v1.1 and GLiDE v1.2. Accuracy did not rise (21% and 9%). Instead, the model refuses to answer more often when it is unsure, so the share of answers without fabrications rose from 85% to 89%. For agents where errors are costly, that is a plus.

AI-processed from X: @ArtificialAnlys; edited by Hamidun News
SubQ Snap 1.2 Rises from 68 to 73 Points on Decision Index: It Says 'I Don't Know' More Often
Source: X: @ArtificialAnlys. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceX: @ArtificialAnlys ↗
Editor-in-Chief
Key metric #1Share of fully correct documents on AA-Paperclip: from 46% to 62%
Key metric #2On AA-Omniscience-Decision questions, the model answers in 29% of cases instead of 33%
Price$0.04 per 1 million input tokens, context window of 384 thousand tokens

Imagine an agent that decides for itself which action to take: approve a payment, reply to a customer, close a ticket. Average accuracy tells you little here. What matters more is that the model does not make up an answer when it doesn't know, and does not change its decision if the same options are simply reworded. Independent lab Artificial Analysis evaluates exactly these properties in its Decision Index. In version v2.1, Subquadratic's SubQ Snap 1.2 scored 73 points against 68 for its predecessor SubQ Snap 1.1. That is a gain of 5 points, and it draws the model level with the leaders of the ranking. But the improvement came from an unexpected place.

What the ranking showed

Two competitors, Drex Max v1.1 and GLiDE v1.2, also scored 73 points, and the gap with SubQ Snap 1.2 is within 0.1 points. In effect, three models are competing for first place, and you cannot choose between them based on the ranking score alone.

The price stayed the same: $0.04 per million input tokens. The context window is 384 thousand tokens. The improvement came without a price increase, which matters if you are counting the cost of processing large volumes of text.

The two subtests that grew the most were AA-Paperclip and AA-Symmetry, with 8 points each.

Where the growth came from

AA-Paperclip tests work with documents. It measures the share of documents the model answered completely correctly, without a single error. For SubQ Snap 1.2, this rose from 46% to 62%. Drex Max v1.1 scores 63% on this measure and GLiDE v1.2 scores 62%, so here all three models are nearly equal.

AA-Symmetry tests stability: does the model keep the same decision and the same confidence when the question changes in a way that should not affect the answer? For example, you ask an agent to choose one of three action options, then restate those options in different words. A reliable model will choose the same thing. In Decision Index v2.1, this test was updated to version 1.1: the answer options now sit inside the description of the situation and are reworded while keeping the same meaning. The task became harder, yet SubQ Snap 1.2 still improved its result by 8 points.

Caution instead of accuracy

The two other tests, AA-Omniscience-Decision and HLE-MultiChoice, are the most interesting. The number of correct answers did not increase: accuracy stayed at 21% and 9%. However, the share of answers without fabrications rose, meaning cases where the model did not give a confidently wrong answer: from 85% to 89% in the first test and from 46% to 55% in the second.

The reason is refusals. On AA-Omniscience-Decision, the model now attempts to answer 29% of questions, down from 33%. It more often admits it doesn't know, and less often offers an incorrect guess. The ranking counts this as an improvement, because a fabricated answer is penalized more heavily than an honest refusal.

What practitioners should do

First, look at what the gain is made of. Five points do not mean the model has become smarter: a noticeable part of the improvement comes from caution. For scenarios where errors are costly (finance, medicine, legal documents, automated agent actions), this is good. If, however, you need some answer to every request, excess caution will get in the way.

Second, measure the refusal rate on your own data. If the model answers 29% of questions instead of 33%, your product's behavior will change. The interface must be able to handle "I don't know": passing the question to a human or to another model.

Third, when scores are equal, compare price, speed and context length. At $0.04 per million input tokens and a 384-thousand-token window, SubQ Snap 1.2 looks like a reasonable choice for bulk processing of long documents.

Limitations: these are benchmarks from a single independent analyst, and the stability test was recently updated, so the numbers cannot be compared directly with past versions of the index. Results on a real task may differ. Before switching to the new version, run your own set of examples.

Story

Key Questions Answered

What the ranking showed?

Two competitors, Drex Max v1.1 and GLiDE v1.2, also scored 73 points, and the gap with SubQ Snap 1.2 is within 0.1 points. In effect, three models are competing for first place, and you cannot choose between them based on the ranking score alone. The price stayed the same: $0.04 per million input tokens. The context window is 384 thousand tokens. The improvement came without a price increase, which matters if you are counting the cost of processing large volumes of text.

Where the growth came from?

AA-Paperclip tests work with documents. It measures the share of documents the model answered completely correctly, without a single error. For SubQ Snap 1.2, this rose from 46% to 62%. Drex Max v1.1 scores 63% on this measure and GLiDE v1.2 scores 62%, so here all three models are nearly equal. AA-Symmetry tests stability: does the model keep the same decision and the same confidence when the question changes in a way that should not affect the answer?

What are the key details on Caution instead of accuracy?

The two other tests, AA-Omniscience-Decision and HLE-MultiChoice, are the most interesting. The number of correct answers did not increase: accuracy stayed at 21% and 9%. However, the share of answers without fabrications rose, meaning cases where the model did not give a confidently wrong answer: from 85% to 89% in the first test and from 46% to 55% in the second. The reason is refusals. On AA-Omniscience-Decision, the model now attempts to answer 29% of questions, down from 33%.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…