Nodul Runs DeepSeek V4.1 Flash on 2 H200s Instead of 8 and Speeds Up Generation to 36.4 Tokens/s
The standard vLLM configuration for DeepSeek V4.1 Flash calls for eight GPUs, but Nodul got the model running on two H200s. It moved rarely used "experts" (the model's subnetworks) into system RAM and kept the frequently used ones in GPU memory. With 20 concurrent sessions, speed rose from 5.4 to 36.4 tokens per second per agent. That matters for teams whose security policies forbid sending code to external clouds.
AI-processed from CNews ИИ; edited by Hamidun News
The Russian company Nodul has run the DeepSeek V4.1 Flash model for coding agents on two Nvidia H200 GPUs, even though the standard configuration of the vLLM engine (the software that runs a model on GPUs) is built for eight. On the same server, generation speed rose from 5.
4 to 36.4 tokens per second per agent with 20 concurrent sessions. A token is roughly a word or a piece of a word, so this is a measure of how fast an AI assistant types out an answer or code.
For teams whose security departments forbid sending source code and documents to external clouds, this offers a chance to run their own server at a third to a quarter of the cost.
Why two cards couldn't hold the model
According to Nodul, it found no ready-made configuration in English- or Chinese-language sources that could serve 20 parallel sessions on two H200s at 20 tokens per second or more per agent. The arithmetic is simple. After the driver and engine started, each H200 had about 139 GB of GPU memory left, while the model weights alone needed about 147 GB. Memory is also needed for context, because each agent session has its own set of files it has read and its own history.
The model's architecture suggested a way out. DeepSeek consists of many subnetworks, or "experts," and calls on them with different frequency. Think of a desk and an archive: the folders you work with every day stay within reach, and the rest go into a cabinet. Nodul did the same. It kept the weights of the most-used experts in GPU memory and moved the rarely used ones into the server's system RAM. The team calls the approach Lossless Expert Tiering, meaning multi-level placement of experts without loss of quality.
How speed grew almost sevenfold
The most useful part of the story is the order of the steps.
The team built the first layout for the modified vLLM from public statistics: it put on the GPU the experts that the model used most often across various tasks. With 20 parallel sessions, speed rose to 17.3 tokens per second per agent.
The team then collected statistics on a workload close to real coding-agent sessions. It turned out that the model favors a different set of experts when working with code. After the team moved those experts' weights to the GPU, the layout change alone added 62%, taking speed to 28.1 tokens per second.
Finally, at 20 sessions the GPU memory reserved for context was less than half used, so the team gave part of the free space to experts. It also taught the engine to change expert placement based on call frequency without a restart. Together, these changes added about another 30% and brought median speed to 36.4 tokens per second per agent. The starting point on the same server was 5.4.
Costs and limitations
For testing, Nodul replayed 1018 requests from real working sessions. With 20 parallel sessions, generation started within 2.5 seconds in nine cases out of ten. With 30 sessions, median speed was 20.1 tokens per second per agent, which leaves the 20-token threshold almost at the limit. All measurements on two H200s were run on a single server.
Nodul's CEO and founder estimates the rental cost of an eight-GPU server at about 4 million rubles a month. For two H200s, the benchmark is 400–500 thousand rubles a month with hourly rental, if you shut the server down at night. For a team of roughly 25–27 people, that comes to about 15–20 thousand rubles per developer. In his words, this option suits small and mid-sized teams that are forbidden to send code and documents to external services.
The caveats matter. The figures come from the company itself, and the article includes no independent measurements. The layout can't be downloaded: the model calls on different experts for different tasks, and a layout built on one workload performs worse on another. If your agents work not only with code but also with documents or analytics, you will have to collect the statistics again.
What to do in practice
For anyone building an internal environment with coding agents, the takeaway is this: before you rent the "required" eight cards, measure exactly how your workload uses the model. For models with experts, you can count call frequency and put it to use: move the rarely used experts to system RAM and free up GPU memory for what is needed right now. The approach requires engine modifications and measurements, but for a team that cannot use an external API, a severalfold saving justifies the effort.
- Nov 19, 2026AI Models Added a School to the Target List in 70% of Vals AI Tests
- Nov 19, 2026The Word "Video Game" Raised GPT-6 Astra's Willingness to Pick Targets From 6% to 92%
- Nov 18, 2026Sciforium pushes DeepSeek-V4.1-Flash on AMD to 1 129 tokens per second
- Nov 18, 2026One Chinese Comma Switched DeepSeek to Chinese in 94% of Responses
Key Questions Answered
Why two cards couldn't hold the model?
According to Nodul, it found no ready-made configuration in English- or Chinese-language sources that could serve 20 parallel sessions on two H200s at 20 tokens per second or more per agent. The arithmetic is simple. After the driver and engine started, each H200 had about 139 GB of GPU memory left, while the model weights alone needed about 147 GB. Memory is also needed for context, because each agent session has its own set of files it has read and its own history. The model's architecture suggested a way out.
How speed grew almost sevenfold?
The most useful part of the story is the order of the steps. The team built the first layout for the modified vLLM from public statistics: it put on the GPU the experts that the model used most often across various tasks. With 20 parallel sessions, speed rose to 17.3 tokens per second per agent. The team then collected statistics on a workload close to real coding-agent sessions. It turned out that the model favors a different set of experts when working with code.
What are the key details on Costs and limitations?
For testing, Nodul replayed 1018 requests from real working sessions. With 20 parallel sessions, generation started within 2.5 seconds in nine cases out of ten. With 30 sessions, median speed was 20.1 tokens per second per agent, which leaves the 20-token threshold almost at the limit. All measurements on two H200s were run on a single server. Nodul's CEO and founder estimates the rental cost of an eight-GPU server at about 4 million rubles a month.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.