Unsloth Updates Compressed Qwen3.8-27B: Accuracy Up 5–10% at the Same Size
To run the large Qwen3.8-27B model at home, its weights are compressed to 2–6 bits, and quality drops as a result. Unsloth has released the Dynamic v3.1 update: on benchmarks including Terminal Bench 2.1 and DeepSWE, the compressed versions answer 5–10% more accurately at the same file size. The 4-bit version fits into 16–19 GB of video memory, enough for an RTX 4090 or a Mac with 24 GB. If you run the model locally for coding and agents, it is worth re-downloading the weights.
AI-processed from Hacker News AI; edited by Hamidun News
Qwen3.8-27B is a 27-billion-parameter Qwen model that writes code well, works as a terminal agent, understands images, and handles dialogue. At full precision it weighs tens of gigabytes and does not fit into the memory of a typical graphics card.
That is why it gets compressed: each weight is stored not in 16 bits but in 2–6. It is like JPEG for a photo: the file is smaller, but details are lost. Unsloth produces such compressed versions, and in the Dynamic v3.
1 update it claims that, at the same file size, average accuracy on most benchmarks rose by 5–10% compared with its previous versions. For those who run the model themselves, this means the same memory footprint and better-quality answers.
What changed in Dynamic v3.1
The update was released on November 11, 2026. The new compression algorithm affected nearly all variants from 2 to 6 bits. The exception is the 3-bit UD3 builds: UD3.1 gave almost no gain on them, so the team decided not to change them.
Testing covered the agentic tasks Terminal Bench 2.1 and DeepSWE v1.1, as well as LiveCodeBench v6, GPQA Diamond, MMLU-Pro and AIME25. In the agentic tests the context was 256K tokens; in the others, 128K. Generation parameters were temperature 1.0, top_p 0.95, top_k 20, with reasoning preserved and maximum reasoning effort.
For practical use, hardware requirements matter most. The 4-bit version runs on 16–19 GB of video memory: RTX 5080, RTX 4090 or a Mac with 24 GB of memory will work. The authors' rule of thumb is that the sum of system RAM and video memory should roughly equal the size of the chosen version. If there is not enough, the model will still launch, but noticeably slower, because some data has to be loaded from disk. It is also worth keeping a 1–2 GB margin.
The dispute with ISTA over how to measure accuracy honestly
A separate part of the material is a debate with the competing GSQ-RCO method from ISTA. In ISTA's comparison, Unsloth's older compressed versions looked worse on LiveCodeBench. Unsloth got the opposite result: UD3 beats ISTA at every size except the smallest. The authors suspect ISTA ran with a context of about 96K rather than 128K. At 96K the GSQ-RCO method is indeed slightly ahead; at 128K, UD3 wins clearly.
The second example is AIME25. ISTA reported that the compressed versions retained 100% of the original model's accuracy in three of four cases and 96.7% in the fourth. Unsloth believes there was only a single run in those cases. The team itself ran 14 passes and got an average accuracy retention of 93–96%, not 100%. Greedy decoding (temperature 0) did not produce stable 100% either: GPU computations are not fully deterministic, and in real use almost no one generates this way.
The practical lesson for anyone comparing models: a single run proves almost nothing. Run the test several times and at the context length you will actually use in your product.
There is also a caveat about Exllama-v3. This library stores its token table in BF16 format. Unsloth removed it from the main size charts but acknowledges that, in terms of real disk and memory usage, the picture looks different. When choosing a version, compare the full file sizes.
NVFP4, reasoning modes and limitations
Owners of Blackwell graphics cards (RTX 50-series, DGX Spark, B200, B300) are offered NVFP4 versions. According to Unsloth, they are 1.5 times faster than BF16 and retain 92–97% agreement with the original on top-1 token prediction. Initially they could only be run in vLLM; later SGLang version 0.5.19 also supported them. For older cards, GGUF files remain available, which work in llama.cpp and in the Unsloth Desktop app.
By default the model reasons at maximum effort. To shorten reasoning chains and token consumption, you can pass the reasoning_effort parameter in llama.cpp, for example medium. There is also a mode that preserves past reasoning between turns: it uses more tokens, but accuracy in long dialogues may improve.
The main caveat: all figures come from Unsloth itself, there is no independent verification yet, and the full tables in the source are reduced to links. However, the authors disclosed their methodology in enough detail that the measurements can be reproduced. If you use Qwen3.8-27B locally for programming or agents, updating the weights to v3.1 is sensible. Meanwhile, test your own scenario over several runs rather than relying on a single successful result.
- Nov 21, 2026Apple Is Building an AI Agent and Making It Harder for Outsiders to Get Access: Permission Now Takes 14 Steps, Not 4
- Nov 21, 2026The Programmer Stopped Being an AI Agent Dispatcher: Multica Takes Over the Handoff Routine
- Nov 21, 2026Developer Skips Pinecone: Searching 72,450 Code Chunks in 50 ms
- Nov 21, 2026Eight Open-Source Tools for AI Agents: Browser Use Leads with 121,000 Stars
Key Questions Answered
What changed in Dynamic v3.1?
The update was released on November 11, 2026. The new compression algorithm affected nearly all variants from 2 to 6 bits. The exception is the 3-bit UD3 builds: UD3.1 gave almost no gain on them, so the team decided not to change them. Testing covered the agentic tasks Terminal Bench 2.1 and DeepSWE v1.1, as well as LiveCodeBench v6, GPQA Diamond, MMLU-Pro and AIME25. In the agentic tests the context was 256K tokens; in the others, 128K.
What are the key details on The dispute with ISTA over how to measure accuracy honestly?
A separate part of the material is a debate with the competing GSQ-RCO method from ISTA. In ISTA's comparison, Unsloth's older compressed versions looked worse on LiveCodeBench. Unsloth got the opposite result: UD3 beats ISTA at every size except the smallest. The authors suspect ISTA ran with a context of about 96K rather than 128K. At 96K the GSQ-RCO method is indeed slightly ahead; at 128K, UD3 wins clearly. The second example is AIME25.
What are the key details on NVFP4, reasoning modes and limitations?
Owners of Blackwell graphics cards (RTX 50-series, DGX Spark, B200, B300) are offered NVFP4 versions. According to Unsloth, they are 1.5 times faster than BF16 and retain 92–97% agreement with the original on top-1 token prediction. Initially they could only be run in vLLM; later SGLang version 0.5.19 also supported them. For older cards, GGUF files remain available, which work in llama.cpp and in the Unsloth Desktop app. By default the model reasons at maximum effort.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.