Live Nov 18 · Cohort 3Nov 18, 19:00 MSK — free hands-on AI agents live workshop · Live Cohort 3 starts Nov 23Join Nov 18 Live →Cohort 3 (Nov 23)
ReleaseInsight · On RadarHabr AI→ original

Qwen on two DGX Spark units runs 34–59% faster in tokens per second and outpaces GLM

A developer lost access to Claude Code and is testing local models on two DGX Spark mini-computers. Qwen 3.8 Flash Next generates 97–117 tokens per second, 34–59% faster than on a single Spark. The wait for the first token at a 128K context drops from 57 to 39 seconds. GLM 5.3 Flash writes more pleasantly but gets stuck in loops. For coding, the author picked Qwen.

AI-processed from Habr AI; edited by Hamidun News
Qwen on two DGX Spark units runs 34–59% faster in tokens per second and outpaces GLM
Source: Habr AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceHabr AI ↗
Editor-in-Chief
Key metric #1Prefill (prompt processing) rose by 47–62%, to 3.1–3.8 thousand tokens per second
Key metric #2At a 256K context, the first token arrives after 86 s with Qwen and 154 s with GLM
GLM90% of its Bash output is comments, and in one loop it created 119 files

What do you do when your cloud coding assistant disappears but you still have to work? The author of a series of notes on Habr lost his Claude Code account and has spent the past two weeks testing whether open-weight local models can replace it. He installed two models, GLM 5.

3 Flash and Qwen 3.8 Flash Next, on two NVIDIA DGX Spark mini-computers joined into a cluster. He measured speed with three numbers.

TPS is how many tokens (chunks of text) the model produces per second. Prefill is how fast it "reads" the prompt it receives, for example an entire repository. TTFT is how many seconds pass before the first token of the answer appears.

Say you ask an agent to find a bug in a large project: the higher the prefill and the lower the TTFT, the less you wait before it starts working. If you are choosing a model for a local agent, look beyond tokens per second to these metrics, and also to how the model behaves in long sessions. The result of the experiment: Qwen became the main tool, and GLM stayed on as a backup.

How it was measured and what came out

Both models ran on the same hardware: two DGX Spark units, the TensorFold engine, and a context of up to 1 million tokens. The quantization, meaning the degree to which the weights are compressed, differs: GLM uses 4 bits per weight (EXL3 format), Qwen uses INT4-AR. At each context size, from 2K to 256K tokens, the author took 10 measurements. The prompts were unique, with a random addition, so the prefix cache did not help: this is an honest "cold" parse of the text from scratch. Each series began with one warm-up request.

On two Spark units, Qwen 3.8 Flash Next generates 97–117 tokens per second, and its prefill is 3.1–3.8 thousand tokens per second. At a 128K context the first token takes 39 seconds, and at 256K it takes 86 seconds. On the same two Spark units, GLM generates 65–72 tokens per second, its prefill is 1.5–1.8 thousand tokens per second, and at 256K the first token takes 154 seconds. Per single Spark, that works out to 33–36 tokens per second for GLM, the worst result among the author's four configurations.

Compared with the single Spark from the previous article, generation is 34–59% faster and prefill is 47–62% faster. TTFT at 128K fell from 57 to 39 seconds (−32%). Speed does not double: the second machine gives roughly a 1.5x gain, not 2x. Prefill scales better than generation, so the cluster is especially useful when an agent frequently re-reads large codebases. In the previous article, DeepSeek on two Spark units produced 99–109 tokens per second, so Qwen is on par with it or slightly ahead, and noticeably ahead on prefill and TTFT.

There is a caveat. The single Spark ran 4-bit NVFP4 quantization, while the cluster ran INT4-AR, so the comparison is not entirely precise and shows only the order of magnitude. There is no 256K measurement for a single Spark.

What is wrong with GLM 5.3 Flash

The paradox is that GLM's answers are more pleasant to read. According to the author, Qwen and DeepSeek sometimes mix up word endings and invent new words, like "echoшить", or insert Chinese characters. GLM writes evenly, moves items on a TODO list neatly at the high and max levels, and follows skills and templates precisely.

For agent work, though, it has some unpleasant habits:

  • 90% of its Bash output is the model's own comments. The context balloons quickly, and the "Thinking" blocks that were present at the start of the session disappear: the reasoning moves inside the commands.
  • Looping. On the very first task, the agent created 119 files named like /tmp/probe119.php. The author noticed only because his plugin displays tool calls. Left unattended, it could have filled the disk by morning.
  • Confusion about its own actions once the context exceeds 130–150 thousand tokens. The model "investigates" who switched the branch, then discovers that its own script, calibrate.sh, did it.

Afterward, the author added a rule to AGENTS.md: if three different attempts produce the same result, stop and rethink the hypothesis instead of launching a fourth. The practical takeaway: an autonomous agent needs a limit on repeated actions, or both tokens and disk space will leak away.

Tooling around the model

The author works through a minimalist harness called Pi in WSL and wrote a plugin for it for PhpStorm and other JetBrains IDEs. Over the week he added three things. The first is Telegram notifications for when the agent finishes a turn, asks for permission, or poses a question.

The second is remote sessions: you can control the agent from a browser on your phone. The third is an ask_question tool, an analog of AskUserQuestion from Claude Code, with the option to add a note to an answer choice or ask for the question to be explained. DeepSeek and Qwen wrote the new features: they rarely got them right on the first try, but by the second or third attempt everything worked.

The practical effect: the agent can run a benchmark overnight without trouble, and you can answer its question at five in the morning from your phone. With no subscription limits, the local agent works around the clock, and the author admits that his dependence on agents has only grown.

It is important to remember that this is one developer's experience on pet projects, not an independent benchmark. He has worked with Qwen only briefly, and there are no figures on code quality. As a next step, the author promises an experiment with the reasoning effort level for coding.

Story

Key Questions Answered

How it was measured and what came out?

Both models ran on the same hardware: two DGX Spark units, the TensorFold engine, and a context of up to 1 million tokens. The quantization, meaning the degree to which the weights are compressed, differs: GLM uses 4 bits per weight (EXL3 format), Qwen uses INT4-AR. At each context size, from 2K to 256K tokens, the author took 10 measurements. The prompts were unique, with a random addition, so the prefix cache did not help: this is an honest "cold" parse of the text from scratch.

What is wrong with GLM 5.3 Flash?

The paradox is that GLM's answers are more pleasant to read. According to the author, Qwen and DeepSeek sometimes mix up word endings and invent new words, like "echoшить", or insert Chinese characters. GLM writes evenly, moves items on a TODO list neatly at the high and max levels, and follows skills and templates precisely. For agent work, though, it has some unpleasant habits: 90% of its Bash output is the model's own comments.

What are the key details on Tooling around the model?

The author works through a minimalist harness called Pi in WSL and wrote a plugin for it for PhpStorm and other JetBrains IDEs. Over the week he added three things. The first is Telegram notifications for when the agent finishes a turn, asks for permission, or poses a question. The second is remote sessions: you can control the agent from a browser on your phone.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Hands-On AI Engineering · Hamidun Academy

Build & Ship Production AI Agents Without Boilerplate Code

Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.

Start 72h Free Trial →Vibe-Coding Intensive
What do you think?
Loading comments…