Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseAPI / Product · ActionableHacker News AI→ original

Unsloth Updates Compressed Qwen3.8-27B: Accuracy Up 5–10% at the Same Size

To run the large Qwen3.8-27B model at home, its weights are compressed to 2–6 bits, and quality drops as a result. Unsloth has released the Dynamic v3.1 update: on benchmarks including Terminal Bench 2.1 and DeepSWE, the compressed versions answer 5–10% more accurately at the same file size. The 4-bit version fits into 16–19 GB of video memory, enough for an RTX 4090 or a Mac with 24 GB. If you run the model locally for coding and agents, it is worth re-downloading the weights.

AI-processed from Hacker News AI; edited by Hamidun News
Unsloth Updates Compressed Qwen3.8-27B: Accuracy Up 5–10% at the Same Size
Source: Hacker News AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceHacker News AI ↗
Editor-in-Chief
The 3-bit versions were left untouchedthe old UD3 performed on par with UD3.1
Unsloth disputed the ISTA testit ran 14 AIME25 passes against ISTA's one
Key metric #3NVFP4 on Blackwell graphics cards is 1.5 times faster than BF16

Qwen3.8-27B is a 27-billion-parameter Qwen model that writes code well, works as a terminal agent, understands images, and handles dialogue. At full precision it weighs tens of gigabytes and does not fit into the memory of a typical graphics card.

That is why it gets compressed: each weight is stored not in 16 bits but in 2–6. It is like JPEG for a photo: the file is smaller, but details are lost. Unsloth produces such compressed versions, and in the Dynamic v3.

1 update it claims that, at the same file size, average accuracy on most benchmarks rose by 5–10% compared with its previous versions. For those who run the model themselves, this means the same memory footprint and better-quality answers.

What changed in Dynamic v3.1

The update was released on November 11, 2026. The new compression algorithm affected nearly all variants from 2 to 6 bits. The exception is the 3-bit UD3 builds: UD3.1 gave almost no gain on them, so the team decided not to change them.

Testing covered the agentic tasks Terminal Bench 2.1 and DeepSWE v1.1, as well as LiveCodeBench v6, GPQA Diamond, MMLU-Pro and AIME25. In the agentic tests the context was 256K tokens; in the others, 128K. Generation parameters were temperature 1.0, top_p 0.95, top_k 20, with reasoning preserved and maximum reasoning effort.

For practical use, hardware requirements matter most. The 4-bit version runs on 16–19 GB of video memory: RTX 5080, RTX 4090 or a Mac with 24 GB of memory will work. The authors' rule of thumb is that the sum of system RAM and video memory should roughly equal the size of the chosen version. If there is not enough, the model will still launch, but noticeably slower, because some data has to be loaded from disk. It is also worth keeping a 1–2 GB margin.

The dispute with ISTA over how to measure accuracy honestly

A separate part of the material is a debate with the competing GSQ-RCO method from ISTA. In ISTA's comparison, Unsloth's older compressed versions looked worse on LiveCodeBench. Unsloth got the opposite result: UD3 beats ISTA at every size except the smallest. The authors suspect ISTA ran with a context of about 96K rather than 128K. At 96K the GSQ-RCO method is indeed slightly ahead; at 128K, UD3 wins clearly.

The second example is AIME25. ISTA reported that the compressed versions retained 100% of the original model's accuracy in three of four cases and 96.7% in the fourth. Unsloth believes there was only a single run in those cases. The team itself ran 14 passes and got an average accuracy retention of 93–96%, not 100%. Greedy decoding (temperature 0) did not produce stable 100% either: GPU computations are not fully deterministic, and in real use almost no one generates this way.

The practical lesson for anyone comparing models: a single run proves almost nothing. Run the test several times and at the context length you will actually use in your product.

There is also a caveat about Exllama-v3. This library stores its token table in BF16 format. Unsloth removed it from the main size charts but acknowledges that, in terms of real disk and memory usage, the picture looks different. When choosing a version, compare the full file sizes.

NVFP4, reasoning modes and limitations

Owners of Blackwell graphics cards (RTX 50-series, DGX Spark, B200, B300) are offered NVFP4 versions. According to Unsloth, they are 1.5 times faster than BF16 and retain 92–97% agreement with the original on top-1 token prediction. Initially they could only be run in vLLM; later SGLang version 0.5.19 also supported them. For older cards, GGUF files remain available, which work in llama.cpp and in the Unsloth Desktop app.

By default the model reasons at maximum effort. To shorten reasoning chains and token consumption, you can pass the reasoning_effort parameter in llama.cpp, for example medium. There is also a mode that preserves past reasoning between turns: it uses more tokens, but accuracy in long dialogues may improve.

The main caveat: all figures come from Unsloth itself, there is no independent verification yet, and the full tables in the source are reduced to links. However, the authors disclosed their methodology in enough detail that the measurements can be reproduced. If you use Qwen3.8-27B locally for programming or agents, updating the weights to v3.1 is sensible. Meanwhile, test your own scenario over several runs rather than relying on a single successful result.

Story

Key Questions Answered

What changed in Dynamic v3.1?

The update was released on November 11, 2026. The new compression algorithm affected nearly all variants from 2 to 6 bits. The exception is the 3-bit UD3 builds: UD3.1 gave almost no gain on them, so the team decided not to change them. Testing covered the agentic tasks Terminal Bench 2.1 and DeepSWE v1.1, as well as LiveCodeBench v6, GPQA Diamond, MMLU-Pro and AIME25. In the agentic tests the context was 256K tokens; in the others, 128K.

What are the key details on The dispute with ISTA over how to measure accuracy honestly?

A separate part of the material is a debate with the competing GSQ-RCO method from ISTA. In ISTA's comparison, Unsloth's older compressed versions looked worse on LiveCodeBench. Unsloth got the opposite result: UD3 beats ISTA at every size except the smallest. The authors suspect ISTA ran with a context of about 96K rather than 128K. At 96K the GSQ-RCO method is indeed slightly ahead; at 128K, UD3 wins clearly. The second example is AIME25.

What are the key details on NVFP4, reasoning modes and limitations?

Owners of Blackwell graphics cards (RTX 50-series, DGX Spark, B200, B300) are offered NVFP4 versions. According to Unsloth, they are 1.5 times faster than BF16 and retain 92–97% agreement with the original on top-1 token prediction. Initially they could only be run in vLLM; later SGLang version 0.5.19 also supported them. For older cards, GGUF files remain available, which work in llama.cpp and in the Unsloth Desktop app. By default the model reasons at maximum effort.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…