Moonshot releases Kimi K3 with 2.8 trillion parameters and a 1M-token context window

Moonshot AI has launched Kimi K3, a model with 2.8 trillion parameters, native vision and a 1M-token context window. The company calls it the first open 3T-class model. It is available in the Kimi app, the Kimi Work desktop client, the Kimi Code terminal tool and through the API. Moonshot promises to publish the full weights later, together with a technical report. The developers themselves add a caveat: overall, K3 still trails the closed leaders, Claude Fable 5 and GPT 5.6 Sol, but by their account it consistently beats the other models they tested.

Architecture: 16 experts out of 896

K3 rests on two architectural updates. Kimi Delta Attention scales attention efficiently across sequence length. Attention Residuals selectively retrieve representations across network depth instead of accumulating them uniformly. On top of that, Moonshot made its Mixture of Experts sparser: each token activates 16 of 896 experts within the Stable LatentMoE framework. At this level of sparsity, routing and load balancing become the main engineering problem, and the team has several answers. Quantile Balancing builds the distribution from quantiles of the router's scores, with no heuristic updates and no sensitive hyperparameter. Per-Head Muon optimizes each attention head independently.

Moonshot estimates that all of this adds up to roughly a 2.5x gain in scaling efficiency over Kimi K2. Quantization-aware training starts at the SFT stage, with weights in MXFP4 and activations in MXFP8. For inference, the company recommends supernodes of 64 or more accelerators. Separately, it has contributed to vLLM a prefix-cache implementation for KDA, which will ship alongside the model.

What Moonshot showed

The emphasis is on long agentic tasks. All figures below come from the company itself; there is no independent verification yet.

• GPU kernel optimization: four tasks, up to 24 hours per model, on Hopper and a third-party vendor's GPGPU. K3 is comparable to Fable 5 (with fallback) and clearly ahead of Opus 4.8, GPT 5.6 Sol and GPT 5.5.
• MiniTriton: a Triton-style compiler with its own tile-level IR on top of MLIR and PTX generation. On the supported roofline tests it performs no worse than Triton and torch.compile, and in places runs faster. nanoGPT training on it converges stably.
• Chip: in a single 48-hour autonomous run, the model designed an accelerator for a nano-model on its own architecture. It measures 4 mm², runs at 100 MHz, uses 1.46 million standard cells and reaches more than 8 700 tokens/s in simulation.
• Science: it reproduced the universal I–Love–Q relations in astrophysics in about two hours, versus one to two weeks of a researcher's work. The task involved 20+ papers, 300+ equations of state and more than 3 000 lines of Python.
• Video editing: the model assembled a teaser from 56 clips on its own, including beat synchronization.

The caveats in the blog post itself matter. A third party tested Fable 5, and the results may have included fallback. In SWE Marathon, fallback triggered on 35% of tasks, which may have lowered its score. For SWE Marathon, Moonshot used its own task branch, recalibrated for H20. Comparisons with competitors under these conditions should be read with caution.

Pricing and access

K3 runs in Kimi.ai, in Kimi Work 3.1.0 and later (Windows and Mac on Apple silicon), in Kimi Code via the /model command, and in the API under the name kimi-k3. Pricing is $0.30 per million input tokens on a cache hit, $3.00 without cache, and $15.00 per million output tokens. Moonshot claims that on the Mooncake architecture, the cache hit rate in coding scenarios exceeds 90%. For long agentic sessions, that is what determines the final bill. At launch, the maximum reasoning mode is on by default; low and high modes will come later. For enterprise teams, Moonshot offers Kimi Enterprise, which separates personal and organizational accounts.

The practical takeaway for developers: if you run long coding sessions, it makes sense to compare K3 with your current model on real repositories and calculate the cost with caching factored in. Those who want to self-host the model will have to wait for the weights and plan for the infrastructure requirements: 64 or more accelerators for comfortable inference.