Moonshot AI’s MoonEP, an MIT-licensed communication library, eliminates load imbalance and boosts scaling efficiency for large Mixture-of-Experts models.
Moonshot AI open sourced MoonEP, an Expert Parallelism communication library designed to optimise distributed Mixture-of-Experts training and inference, under the MIT licence on 29 July 2026. The library was released as part of Kimi K3 Open Day alongside the 2.8-trillion-parameter Kimi K3 model weights, technical report, and companion infrastructure codebases FlashKDA and AgentEnv. MoonEP served as a key architectural component in achieving a claimed 2.5-fold improvement in scaling efficiency for the Kimi K3 model.
MoonEP guarantees a strict structural invariant where every processing rank receives a perfectly predictable workload regardless of routing imbalance. Load balancing is achieved dynamically by using a fast GPU planning component to determine and prefetch redundant experts online from current router outputs.
The system transfers workload data directly into expert-grouped remote rank positions without intermediate communication-buffer copying. By maintaining static processing shapes across iterations, MoonEP eliminates per-layer host synchronisation and prevents GPU memory fragmentation.
Framework integration requires a contiguous symmetric-memory weight layout per expert projection, allocating dedicated prefetch slots for duplicated experts. Training workloads reserve prefetch capacity bounded by the total expert count divided by the number of parallel processing ranks, whereas inference workloads can operate with a small fixed number of slots while handling overflow via direct symmetric-memory reads. During the training backward pass, gradient updates for duplicated experts are handled through a dedicated reduce buffer and mapped over high-speed interconnects before accumulating into local parameter gradients.
In benchmark comparisons against DeepEP v2 on NVIDIA H20 GPUs, MoonEP maintained nearly constant communication time as router imbalance scaled. Eliminating user-buffer copying allowed Moonshot AI’s MoonEP to consistently beat DeepEP v2 in raw communication latency across all tested imbalance levels.
