Moonshot open-sources Kimi K2: a 1-trillion-parameter MoE built for agents

Moonshot AI has released the weights of Kimi K2, a Mixture-of-Experts model with 1 trillion parameters, of which 32 billion are active for each token. It comes in two versions: Kimi-K2-Base for those who want to fine-tune the model themselves, and Kimi-K2-Instruct for chat and agentic scenarios. Instruct does not reason at length and answers immediately. The 0905 update followed later, with improved agentic coding and a 256K context window. According to the company, K2 delivers the best results among non-thinking models in knowledge, math, and coding.

MuonClip: how to train a trillion parameters without failures

The main technical news here is not the size but the optimizer. The Moonshot team has long bet on Muon. In its earlier work, Moonlight, Muon clearly beat the standard AdamW in token efficiency. For K2, the team scaled this approach to an architecture close to DeepSeek-V3. Based on scaling-law analysis, the engineers reduced the number of attention heads to improve long-context efficiency and increased MoE sparsity to improve token efficiency.

At large scale, a problem appeared: Muon is more prone to exploding attention logits. Logit soft-capping and query/key normalization did not help. The fix was qk-clip. After each Muon step, the weights of the query and key projections are rescaled by a factor η. It equals min(t / max(qᵀk), 1), where t is set in advance. This caps the logit scale at its source rather than after the problem appears. The result: pretraining on 15.5 trillion tokens ran without a single loss spike. Moonshot separately notes that the technique is general and may be useful for other stabilization problems.

Where the agentic skills come from

The model was taught to act in two stages. First, the team built a synthetic data pipeline modeled on ACEBench. It spans hundreds of domains and thousands of tools, both real MCP tools and synthetic ones, and generates hundreds of agents with different tool sets. The agents interact with a simulated environment and simulated users. Tasks are scored against rubrics, and an LLM judge filters out weak trajectories. What remains goes into rejection sampling and RL.

The second stage is general RL, where the hardest tasks are those without a verifiable reward, such as writing a research report. Here the model acts as its own critic and scores results against rubrics. The critic is continuously tuned on on-policy rollouts with verifiable rewards (math, competitive coding) so that its evaluation keeps pace with the policy.

In the demos, K2 chooses tools on its own. A salary analysis took 16 IPython calls. A plan for a trip to Coldplay in London was assembled in 17 calls across search, calendar, Gmail, flights, Airbnb, and restaurants. In Flask-to-Rust, the model rewrites a project and runs benchmarks.

Key specs

• Architecture: MoE, 1 trillion parameters, 32 billion active
• Pretraining: 15.5 trillion tokens, MuonClip optimizer, no failures
• Release: Kimi-K2-Base and Kimi-K2-Instruct
• Context after the 0905 update: 256K
• Inference engines: vLLM, SGLang, KTransformers, TensorRT-LLM
• API: compatible with OpenAI and Anthropic interfaces

What to try and where the pitfalls are

For a quick test, the model is available for free in the web and mobile Kimi apps. MCP features are still under development there, and vision is not supported. Developers will find it easier to connect through the API, which accepts OpenAI and Anthropic clients: you will hardly need to change your application code, just swap the endpoint and model. For your own hardware, the authors recommend the engines listed above and point to instructions on GitHub. A trillion parameters requires a serious cluster, so self-hosting currently suits only teams with ready infrastructure.

Moonshot itself names the limitations. On hard reasoning tasks or when tool descriptions are vague, the model may generate excess tokens, and then the response gets cut off or a tool call is left incomplete. With tools enabled, quality drops on some tasks. A whole software project from a single prompt comes out worse than in an agentic harness. The practical takeaway: set token limits, describe tool schemas clearly, and give K2 a loop with tests and logs rather than one big request. The company promises to add a thinking mode and vision later.