An open-source training kernel combines communication and computation into a single execution pipeline, helping accelerate the training of large mixture-of-experts artificial intelligence models on high-performance GPU clusters.
Cursor Research has released Mixture-of-Kittens (MoK) as an open-source training kernel for mixture-of-experts (MoE) models. Designed for large NVIDIA NVL72 GPU systems, the project combines all communication and computation steps of MoE training into a single deterministic kernel, reducing overhead during large-scale model training.
Unlike traditional MoE implementations, which process communication and computation separately, MoK combines them into a single execution pipeline. This method increases data transfers between GPUs, avoids any additional synchronization between the CPU and GPU, and provides deterministic execution, ensuring repeated training runs produce identical results.
According to Cursor, the training kernel achieves 2.37× higher forward-pass throughput than the best existing public baseline on GB300 NVL72 hardware platforms, in addition to increasing the overall training performance on GPU clusters. The software is currently being used to train the company’s Composer models on GPU clusters numbering in the tens of thousands.
The project is released under the Apache 2.0 licence and, allowing developers to inspect, modify, and extend the source code. The system requires NVIDIA Blackwell SM100 or SM103 GPUs along with CUDA 13.0, Python 3.12 or newer and PyTorch 2.10 or newer, which means that it is primarily targeted at organisations with extensive AI computing capacity.
By making its production training kernel publicly available, the project provides researchers and infrastructure developers with a high-performance foundation for developing large mixture-of-experts models.
