Kimi K3.1 Pro Weights Released Openly: 2.8 Trillion Parameters, 104 Billion Active

Sometimes the most important news arrives as a single line on X. The post by @braincramps got just 13 views, but it is packed with specifics: the Kimi K3.1 Pro weights have been published, it is an MoE with 2.8 trillion parameters and 104 billion active, 93 layers, a custom license, and the model is multimodal. The author calls it the largest open release of the week. The post offers no confirmation from other sources and no benchmark results, so we treat it as a claim to be verified.

What These Numbers Mean

An MoE (mixture of experts) works like this: the model has a huge pool of "experts," but only a small portion is activated for each token. Here that is 104 billion out of 2.8 trillion, or about 3.7% of the parameters. In terms of per-token compute, the model is comparable to a dense model of just over 100 billion parameters. But everything still has to be held in memory. Simple arithmetic: in 8-bit format the weights take roughly 2.8 TB, in 4-bit about 1.4 TB. This cannot be run on a single multi-GPU machine; you need a multi-node cluster.

Hence the practical takeaway. "Open weights" at this scale do not mean "I'll put it on my server." For most teams there is only one realistic path: wait until hosting providers deploy the model and access it via API.

Architecture: A Hybrid Instead of Classic Attention

Of the 93 layers, 69 are KDA and 24 are gated DSA. That means roughly three quarters of the stack consists of one layer type and about a quarter of the other. The post does not spell out the abbreviations. KDA, judging by the name, seems to refer to the linear attention from the Kimi line (Kimi Delta Attention), and DSA to sparse attention. That is our guess, not a fact from the source.

The reason for this design is well known. Classic attention gets more expensive as context length grows and bloats the KV cache. Hybrids with linear and sparse layers compress that cache and make long context cheaper. If the scheme works in K3.1 Pro, those who run long documents, codebases, and agent chains, where context grows at every step, stand to benefit.

Multimodality and the License

It is also a vision-language model. For product teams, this means screenshots of interfaces, documents, and diagrams can be given to the same model as text, with no separate pipeline. How well it actually does this is not yet known.

The license is labeled "custom," and this is the most important thing the post leaves out. Such licenses may include restrictions: revenue or user-count thresholds, attribution requirements, bans on certain use cases. Until you have read the text, you cannot call the model open source in the strict sense. It is more accurate to speak of open weights.

What to Do in Practice

First, do not rewrite your stack around the model based on one post. Wait for independent measurements on your tasks: code, tool use, long-context work.

Second, check support in vLLM and SGLang. New hybrid architectures often reach inference engines with a delay, and that determines when the model actually becomes available.

Third, have a lawyer read the license, or at least read it carefully yourself, before building the model into a commercial product.

Finally, do the economics. With 104 billion active parameters, the per-token price from a provider may turn out to be reasonable. But the memory requirements will make hosting expensive for those who decide to run the model themselves. For now, the main takeaway is this: Chinese labs keep releasing flagship-scale weights, and that once again raises the bar for open models.