Moonshot AI Open-Sources MoonEP, the Library Used to Train the MoE Model Kimi K3

On July 29, 2026, Moonshot AI released the source code of MoonEP, an Expert Parallelism library for distributed training of MoE models, under the MIT license. MoonEP gives every compute rank exactly S × K tokens regardless of routing skew, and it was one of the technologies that sped up the scaling of the Kimi K3 model by 2.5x.

Why expert parallelism is needed

In MoE models, the router sends each token to several top-K experts that physically live on different GPU ranks. The problem is that routers are almost never balanced: some experts receive several times more tokens than others. The MoonEP repository measures the skew with the metric maxvio = max_e (T_e / T̄) − 1, where zero means perfect balance.

The costs of imbalance are structural, not random. The latency of a collective operation is set by its slowest participant, so the "hot" rank determines the time of the entire iteration. Worse, the number of tokens per rank changes at every step — dynamic shapes fragment GPU memory and force a host synchronization at every layer.

How MoonEP works

MoonEP enforces a hard invariant: every rank receives exactly S × K tokens, where S is the number of input tokens per rank and K is the number of top-K experts per token. The library achieves this by scheduling a small number of redundant (duplicate) experts online, directly from the current router outputs; the duplicates are prefetched before the main computation, and on the backward pass their gradients are reduced back to the "home" ranks. The design rests on three properties: perfect balance, online scheduling with near-zero overhead, and zero-copy with static buffer shapes.

"Every rank receives exactly S × K tokens, no matter how skewed the routing is," says the

MoonEP repository description.

Key facts about the release:

• Date — July 29, 2026, as part of Kimi K3 Open Day
• License — MIT
• Alongside MoonEP, AgentEnv and (earlier) FlashKDA were also open-sourced — three infrastructure codebases in total
• Kimi K3 is an MoE model with 2.8 trillion parameters, native vision, and a 1 million-token context
• The scheduler is written in CUTLASS CuTe DSL; setup.py pins nvidia-cutlass-dsl==4.4.2
• For training, B = E/R is set; for inference, the README recommends B = 3–4

Why this matters to engineers

MoonEP removes the main bottleneck of distributed MoE training — rank desynchronization caused by uneven routing. Thanks to statically known buffer shapes and a zero-copy approach (tokens are written straight into positions grouped by expert), the library eliminates per-layer host synchronization at every step. According to Moonshot AI, it was this set of techniques that produced the claimed 2.5x scalability gain on the 2.8-trillion-parameter model.

What this means

Open-sourcing the infrastructure layer, not just the model weights, lowers the barrier to training very large MoE models: teams outside the top labs get a ready-made expert-balancing mechanism under the permissive MIT license.

Frequently asked questions

What is MoonEP?

MoonEP is an expert-parallelism library from Moonshot AI for distributed training and inference of MoE models. Its main guarantee is exactly S × K tokens per rank regardless of routing skew.

What license is MoonEP released under?

MoonEP is distributed under the MIT license — one of the most permissive licenses, allowing commercial use and modification of the source code.

Which release is MoonEP tied to?

MoonEP was published as part of Kimi K3 Open Day, together with the Kimi K3 model weights, a technical report, and two more codebases — AgentEnv and FlashKDA.