DeepSeek-V4.1-Flash cuts HBM KV cache to 890 bytes per token, down from 3514

DeepSeek has introduced V4.1-Flash, a multimodal MoE model with a 1M-token context. Its main trunk has 552B parameters, and another 196B sit in Engram memory tables. The headline result concerns serving: the global KV cache in HBM is squeezed to 890 bytes per token, whereas the previous DeepSeek-V4-Flash needed 3514 bytes. The long-term SSD cache shrank to roughly one eighth of its previous size. For agent sessions of a million tokens, this removes the main bottleneck: cache memory no longer determines how many sessions a server can handle.

A post in the Telegram channel gonzo_ML points to a paper with roughly 500 authors (arXiv 2609.19969), a review on Substack and the weights on Hugging Face. The code is not publicly available; the model card lists N/A.

Three memory-saving mechanisms

The first mechanism is a causal encoder-decoder scheme, CED. The model departs from the usual decoder-only design: during prefill, when it reads a long prompt, 8B parameters are active, and when generating a response, 16B. The review's author notes that this appears to be the only top open model with such an architecture. The closest analogues are the encoder-decoder T5Gemma and the decoder-decoder YOCO scheme.

The second mechanism is CSA2 attention. It reuses the cache in FP4 format across layers, so each layer does not need to store its own copy. This is where most of the compression down to 890 bytes per token comes from.

The third mechanism is bounded sliding-window replay, SWA Bounded Replay. The local sliding-window cache is no longer written to disk; when needed, it is restored by recomputation over a bounded span. This removes some of the load on the SSD and explains why the disk cache volume dropped roughly eightfold.

Benchmarks: parity with Opus-5 on agentic tasks

On DeepSWE v1.1 (a set of bug-fixing tasks in real repositories), V4.1-Flash solves 74.2% of tasks. V4-Flash scores 54.4% and Opus-5 74.0%. On Terminal-Bench 2.1 (command-line tasks for an agent), the model reaches 90.6% Pass@1, meaning it solves the task on the first attempt. V4-Flash gets 82.7% here and Opus-5 gets 89.1%.

The update, then, not only made inference cheaper but also raised quality in coding and terminal scenarios. The gain on DeepSWE is about 20 percentage points over the predecessor. For developers building long agentic loops, this is a notable argument in favor of an open model.

What the paper does not include

There are limitations. The paper has no per-component ablations, so it is unclear how much CED, CSA2 and window recomputation each contribute to the memory savings and to quality. There are also no retrieval-quality curves for the 1M-token context. The claimed 890 bytes per token show the storage cost, but do not prove that the model reliably finds the right fragment across the full length.

On hard scientific tasks the result is noticeably more modest. On Terminal-Bench 4.0, V4.1-Flash scores 31.2%, while Opus-5 scores 51.8%. Parity with the closed system is confirmed on Terminal-Bench 2.1 and DeepSWE v1.1, but not on the hardest of the listed benchmarks.

The practical takeaway for those who deploy models themselves: according to the paper, at a context of a million tokens the constraint becomes compute, not HBM or SSD for the cache. This changes the cost calculation for long agentic sessions. But until the code, ablations and independent long-context measurements are released, the figures remain the authors' claims. The weights can be downloaded and tested on your own tasks.