Qwen3.8-27B at IQ3 on an RTX 5070 Ti scores 91.1% and hits 68.3 tok/s

A Habr author ran Qwen3.8-27B at IQ3 quantization on a single RTX 5070 Ti and got 91.1% of tasks solved on his own task set. The composite index is 82.3, and generation speed is 68.3 tokens per second in a single stream at 131K context. This is the first part of a series: the local setup will then be compared with Russian cloud providers on quality and on price in rubles.

What was measured

The announcement ties the numbers to "my tasks." That means this is the author's own benchmark, not a public test of the kind labs run. For practical purposes that is even more useful: your own tasks show more accurately whether a model fits a specific job. But the result cannot be compared directly with leaderboard tables.

Final run parameters:

• Model: Qwen3.8-27B, IQ3 quantization
• Hardware: one RTX 5070 Ti
• Share of tasks solved: 91.1%
• Composite index: 82.3
• Speed: 68.3 tok/s, single stream
• Context: 131K tokens

Why IQ3 and 131K are controversial

The author writes that he has heard plenty of opinions like "131K is too little" and "IQ3 is dumb." The objections are understandable. IQ3 compresses weights to roughly three bits per parameter, and this kind of quantization has a reputation for degraded quality. A 131K context also looks modest next to models with windows of hundreds of thousands or millions of tokens.

The explanation lies in the hardware. The RTX 5070 Ti has 16 GB of GDDR7 video memory. A rough estimate: 27 billion parameters at 3–3.5 bits per weight come to about 10–11 GB for the weights alone. The remainder goes to the KV-cache, which grows with context length. This is my own estimate, and the exact figure depends on the model architecture and on whether the cache is quantized. But it shows why the "IQ3 plus 131K" combination does not look accidental. A larger quant would push the long context out of memory.

A speed of 68.3 tokens per second is faster than reading pace and is enough for interactive work: chat, code edits, short agent loops.

What the announcement leaves out

Now for the weak spots. The teaser does not say what the task set contains, how the 82.3 index is calculated, or how it differs from the 91.1% success rate. The runtime, sampling parameters, and method of checking answers are not given. It is unclear how the model behaves with a full context rather than on short prompts. And finally, the 68.3 tok/s figure is for a single stream. For a team whose requests arrive in parallel, a cloud service may win on throughput.

That is exactly why the second part is more interesting. It promises a comparison with a Russian cloud, meaning quality plus cost in rubles.

How to use this data

The author's numbers are better treated as a methodology than as a verdict.

• Assemble 30–50 tasks from your real work: code, document analysis, data extraction. Define in advance what counts as success.
• Run one model at several quantization levels, for example IQ3 and something heavier, and in the cloud. Record the success rate and speed.
• Test long context separately: fill the window to 100K or more and check how quality drops.
• Count the money honestly. On one side are the price of the card, electricity, and your time spent on maintenance. On the other are rubles per million tokens and your actual traffic volume.

If you have a single user and your tasks fit within 131K, a local 16 GB card, judging by this data, may handle most of the work. For several parallel streams, draw conclusions only after the cloud figures in part two.