Local Qwen on an RTX 5080 jumps from 33% to 88.5% on a lab notebook test

When a local neural network loses to a cloud one, the hardware usually takes the blame: not enough VRAM. The author of a series of Habr articles checked whether that is true. His workstation, with a single RTX 5080 (16 GB of VRAM and 62 GB of RAM), posted a modest result in earlier tests.

On questions about an electronic lab notebook, the early Qwen 3.8 IQ2_XS answered only 8 of 24 correctly, or 33%. After the fixes, on the same computer, Qwen 3.

8 IQ3_XXS scored 23.0 points out of a possible 26 (88.5%) on an updated set of 26 tasks and shared first place with Claude Opus 5.

5. Imagine you run a local model to analyze code or data and it seems to "act dumb." The cause may be that the server has quietly switched off the conversation cache, while a token limit cuts the model's reasoning off mid-phrase.

Below is what exactly had to be fixed.

The full build instead of the trimmed one

The Qwen 3.8 Coder IQ1_M build was shrunk to 23.4 GB in two ways. The weights were compressed to roughly 1.6 bits, and half of the routed experts were cut from every layer, leaving 256 of 512. Here, "experts" are separate blocks of the model among which it distributes the work. On simple code edits the losses are barely visible. They show up when the model has to keep several subsystems in mind at once: nested FireDAC transactions, interface recalculation, repacking of project archives, and a background update check with SHA-256 verification. In those cases the trimmed model missed again and again.

The author switched to the full 512-expert build, Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS from the ISTA-DASLab lab (2.84 bits per weight, 40 GiB of experts). By the accuracy-retention tables, it is only 2.3 percentage points better than IQ2_XS: 87.3% versus 85.0% of the uncompressed model. But in a multi-step agent with a context of about a hundred thousand tokens, small inaccuracies accumulate from step to step, and the difference becomes noticeable.

Two traps in the server settings

At first the agent still ran into the 45-minute timeout. The logs of the Strata engine revealed two causes.

First: double memory reservation. The server loaded the most frequently used expert slots (7.0 GiB) into VRAM and at the same time tried to pin all 40 GiB of experts in RAM. That left 0.7 GiB of free RAM, so the server silently disabled the conversation cache. Without it, at every step of a long session the agent re-read 60–100 thousand tokens of history and lost about a minute each time. The --resident-experts flag pins in RAM only the part that does not fit in VRAM, which is 33.5 GiB. The freed 5.4 GiB went to the session cache.

Second: truncated reasoning. Before editing code, the model writes a plan of 6–9 thousand tokens inside a think tag. The agent had a maxTokens ceiling of 8192, and the server had answer_room mode enabled. When the reasoning reached 6144 tokens, the server forcibly closed the tag and left 2048 tokens for the answer. The thought broke off mid-sentence, the model produced a comment without calling any tools, and everything started over. The author raised the limit to maxTokens 16384 and set answer_room: false. The idle loops disappeared, and the model sustained 70–81 tokens per second at a context of 131 072 tokens.

One small note for Windows: pinning 33.5 GiB of memory at startup takes about 30 seconds. So it is better to keep the server running as a single persistent process for the whole series of tasks.

The results

On the 20 Delphi tasks, a task counted as solved if the project builds in RAD Studio with no new compiler warnings and passes the hidden DUnitX tests. The full Qwen without an additional skill solved 19 of 20 (95%), the same as Claude Sonnet 5.5, and passed all the hidden tests. It solved all three tasks on which the trimmed build had stumbled. With the delphi-development skill attached, the result fell to 17 of 20. The author attributes this to statistical scatter from single runs: a strong model reads neighboring modules and picks up the project's style even without hints.

In the ELN Suite 1.1 test (26 tasks, live MCP server), the local model scored 23.0 points (88.5%) and finished in 46 minutes. Claude Sonnet 5.5 and Gemini 4 got 21.0 each, and DeepSeek V4.1 Flash got 19.0. The first run scored 22.0: the model's only zero came on a warm-up question, where it passed a boolean parameter as the string "True" instead of true. The OMP agent shell forgives this. The author's own strict validator and one of the five duplicate parameter parsers in the Delphi server do not.

Practical takeaways and caveats

• Don't use expert-trimmed builds for complex agentic tasks.
• Watch the balance between RAM and the conversation cache: a silently disabled cache turns a session into endless re-reading of context.
• Leave a reasoning model headroom in maxTokens and don't enable forced closing of think.
• Make tools tolerant of "True" and null in optional fields.

The caveats matter. The tests are the author's own, run on private projects, and some configurations were run only once. The old 33% comes from a set of 24 questions, while the new 88.5% comes from an updated set of 26 tasks. The local model is noticeably slower: 46 minutes versus 17 for Opus 5.5 and 11 for Sonnet 5.5. On the other hand, the run costs $0 versus $5.37 for Opus. If the tasks aren't urgent and the data can't leave the lab, that is a serious argument for running your own hardware.