Sciforium pushes DeepSeek-V4.1-Flash on AMD to 1 129 tokens per second
Sciforium has sped up DeepSeek-V4.1-Flash on four AMD MI355X GPUs. On real traffic, throughput rose from 835 to 1 129 tokens per second (P90). The gain comes from speculative decoding: a small draft model guesses several tokens ahead, and the large model verifies them all at once. Sciforium fine-tuned the drafter on actual customer requests and lengthened its guess from 5 to 7 tokens. Bottom line: tailoring the drafter to your own traffic produced roughly 35% more speed without changing the weights.
AI-processed from X: @0x0SojalSec; edited by Hamidun News
Response speed is money. The faster a GPU produces tokens, the fewer cards you need for the same traffic, and the less often users stare at a blinking cursor. That makes the latest result from the startup Sciforium worth a look for anyone running large models in production: its build of DeepSeek-V4.1-Flash generates 1 129 output tokens per second on four AMD Instinct MI355X accelerators. The same setup previously delivered 835.
What exactly was sped up
Here are the key details from the company's announcement. The measurement is P90. That is not a best case, but a figure that holds for the vast majority of requests. The traffic is real customer traffic, not a synthetic run. The model runs with its original weights: it was neither distilled nor pruned. The gain was roughly 35%.
The authors also point to the wider context: not long ago, many doubted that AMD graphics cards could handle heavy LLM workloads in production on par with Nvidia. They present this case as evidence that, with good engineering, the gap closes.
How the drafter trick works
The speedup rests on speculative decoding. Imagine an experienced editor proofreading a trainee's text. Writing from scratch is slow, but checking a finished paragraph is fast: the editor glances over several phrases at once and approves them in one go. Here the trainee is a small draft model (the drafter) that proposes several tokens ahead. The editor is the large DeepSeek-V4.1-Flash, which verifies all the proposed tokens in a single pass. If the guesses are right, one step of the large model yields several tokens instead of one. If they are wrong, the excess is discarded, and the quality of the answer stays the same as that of the large model.
The whole gain depends on how often the drafter guesses correctly. Sciforium calls its approach traffic-tuned speculators: the drafter was fine-tuned on real customer data, meaning the requests and responses the model actually sees. A generic drafter predicts general text well but stumbles on specifics. If your traffic consists of templated support replies, code in a particular stack, or structured JSON, a drafter trained on it hits the target noticeably more often.
The second lever is guess length. It was raised from 5 to 7 tokens at a time. This works only if the drafter is highly accurate: the longer the chain, the greater the risk that it makes a mistake somewhere near the end and the entire tail is lost. The longer guess paid off precisely because the drafter now matched the traffic better.
The third layer is deep optimization of kernels for AMD, that is, the low-level code that performs the computations on the MI355X. Judging by the post, the team stresses that it did not have to rewrite the vLLM engine: the effort went into the drafter and the kernels.
What to take from this
First: if you host an open model yourself, tuning speculative decoding to your own traffic can yield tens of percent more speed without losing quality and without changing the weights. That is cheaper than buying more GPUs.
Second: this requires logs of real requests. Teams that collect them and know how to fine-tune a small model gain an edge over those who use a generic drafter out of the box.
Third: AMD is becoming a workable option for inference rather than an experiment. For founders, that is leverage in price negotiations for compute and a way to avoid depending on a single supplier.
Limitations
This is the company's own claim, and there are no independent measurements yet. The result applies to a specific setup: DeepSeek-V4.1-Flash, four MI355X, and Sciforium's customer traffic. On a different workload, the gain will differ: the share of successful drafter guesses varies across conversational chat, code, and long reasoning. The post does not disclose all the measurement conditions, so be cautious when comparing these numbers with other people's benchmarks. The direction itself is clear and reproducible, though: train the drafter on your own data, tune the guess length, and optimize kernels for your hardware.
- Nov 18, 2026One Chinese Comma Switched DeepSeek to Chinese in 94% of Responses
- Nov 18, 2026AI Finds Unknown V8 Vulnerability for $4.24 in 13 Minutes, After Failing for $350 in February
- Nov 18, 2026AI Agents Shipped 228 Game Releases Without Human Review; Tests Sent Back 25% of Tasks
- Nov 18, 2026Tencent Hy4 flash scores 2.79 out of 4 vs. 2.81 for Haiku 5.5 and costs 18 times less per task
Key Questions Answered
What exactly was sped up?
Here are the key details from the company's announcement. The measurement is P90. That is not a best case, but a figure that holds for the vast majority of requests. The traffic is real customer traffic, not a synthetic run. The model runs with its original weights: it was neither distilled nor pruned. The gain was roughly 35%. The authors also point to the wider context: not long ago, many doubted that AMD graphics cards could handle heavy LLM workloads in production on par with Nvidia.
How the drafter trick works?
The speedup rests on speculative decoding. Imagine an experienced editor proofreading a trainee's text. Writing from scratch is slow, but checking a finished paragraph is fast: the editor glances over several phrases at once and approves them in one go. Here the trainee is a small draft model (the drafter) that proposes several tokens ahead. The editor is the large DeepSeek-V4.1-Flash, which verifies all the proposed tokens in a single pass.
What to take from this?
First: if you host an open model yourself, tuning speculative decoding to your own traffic can yield tens of percent more speed without losing quality and without changing the weights. That is cheaper than buying more GPUs. Second: this requires logs of real requests. Teams that collect them and know how to fine-tune a small model gain an edge over those who use a generic drafter out of the box. Third: AMD is becoming a workable option for inference rather than an experiment.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.