GPT-6 Astra Loses 5 Points at Medium Effort but Solves Tasks Three Times Faster
Most labs set the API reasoning effort level below maximum, and this sharply cuts time per task. According to Artificial Analysis, GPT-6 Astra at medium effort scores 52 points on the Intelligence Index versus 57 at maximum, but completes a task in ~5.8 minutes instead of ~18. For GPT-6.1 Sol, the index drops from 55 to 46 when effort is lowered.
AI-processed from X: @ArtificialAnlys; edited by Hamidun News
A familiar situation: you plug in a top-tier model, run a couple of tests, and the agent thinks for minutes on end. The user closes the tab before it finishes. The server isn't necessarily to blame. More often the cause is a setting many people never touch: the reasoning effort level, meaning how much "thinking time" the model spends before answering.
Analysts at Artificial Analysis reminded readers on X that most leading labs set the API default below maximum effort. This sharply reduces Time per Task, meaning the time the model spends on a whole task. Below, we go through what their numbers showed and what to do with them.
What the numbers showed
First example: GPT-6 Astra. At medium effort the model scores 52 points on the Intelligence Index, and 57 at maximum. The difference is five points, or about 9% of the best result. In return, a task at the medium level takes about 5.8 minutes, versus about 18 at maximum. Time drops roughly threefold, while quality fell by less than a tenth.
Second example: GPT-6.1 Sol. Here the picture is harsher: the index falls from 55 to 46, a drop of 9 points, or about 16%. The post cuts off at the figure 3.9, so the unit of measurement and the exact effort level are not visible in this fragment. Conclusions about Sol should be drawn after viewing the full table via the link in the original.
Why this matters in practice
Imagine a code review agent. It receives a pull request, reads the diffs, runs checks, and writes comments. If each run takes 18 minutes, the developer has already switched to something else, and the comment arrives to no one. At 6 minutes it is still tolerable. For such a scenario, five lost points in the overall index often don't matter, while a threefold lower latency is noticeable immediately.
The opposite case: overnight batch analysis of complex incidents or generating a migration on a large codebase. Nobody is waiting at the screen, and every mistake is expensive. Maximum effort is justified here, because time costs almost nothing while the price of an error is high.
Hence a simple rule: the effort level is a product knob, not a technical detail. It should be chosen for the scenario, not left as it came out of the box.
What to do about it
First: find out which default your provider uses. It differs between labs, and developers often sit for years on a value nobody consciously chose.
Second: measure not only quality but also time per task on your own set of tasks. The overall index is an average across benchmarks, and your traffic may behave differently. Run the same 50–100 real requests at several effort levels and compare the success rate and median time.
Third: consider routing. Send simple requests to low or medium effort, and escalate complex or disputed ones to maximum. That way you get speed in most cases and quality where it is really needed.
Limitations
This is a short post on X, not a full report. We see only two models, and part of the Sol data is cut off. The Intelligence Index is an aggregated metric, and on narrow tasks the gap between effort levels may be noticeably larger or smaller. In addition, time depends on the load on the provider's infrastructure, and your numbers will differ.
Bottom line: don't treat maximum effort as the "right" default mode. For Astra, trading five points for a threefold speedup looks worthwhile for interactive scenarios. For Sol the price is higher, and checking on your own data is especially important.
- Nov 12, 2026Claude in Chrome requires approval for every click on category 3 sites
- Nov 12, 2026Norwest raises record $4 billion fund, betting on agents, voice and physical AI
- Nov 12, 2026AWS shows how 100% retrieval precision can hide an AI agent's failure
- Nov 12, 2026Garage Eight dropped peer approvals for 69 MRs in an internal service using AI review
Key Questions Answered
What the numbers showed?
First example: GPT-6 Astra. At medium effort the model scores 52 points on the Intelligence Index, and 57 at maximum. The difference is five points, or about 9% of the best result. In return, a task at the medium level takes about 5.8 minutes, versus about 18 at maximum. Time drops roughly threefold, while quality fell by less than a tenth. Second example: GPT-6.1 Sol. Here the picture is harsher: the index falls from 55 to 46, a drop of 9 points, or about 16%.
Why this matters in practice?
Imagine a code review agent. It receives a pull request, reads the diffs, runs checks, and writes comments. If each run takes 18 minutes, the developer has already switched to something else, and the comment arrives to no one. At 6 minutes it is still tolerable. For such a scenario, five lost points in the overall index often don't matter, while a threefold lower latency is noticeable immediately. The opposite case: overnight batch analysis of complex incidents or generating a migration on a large codebase.
What to do about it?
First: find out which default your provider uses. It differs between labs, and developers often sit for years on a value nobody consciously chose. Second: measure not only quality but also time per task on your own set of tasks. The overall index is an average across benchmarks, and your traffic may behave differently. Run the same 50–100 real requests at several effort levels and compare the success rate and median time. Third: consider routing.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.