Moonshot AI released Kimi K3, a 2.8 trillion-parameter open-weight model built on a Mixture-of-Experts architecture with 896 experts, incorporating innovations called Kimi Delta Attention and Attention Residuals. The model uses quantization-aware training with MXFP4 weights and MXFP8 activations, cutting storage to roughly 1.4 terabytes and enabling deployment on modest multi-node GPU clusters rather than requiring hyperscale infrastructure. Benchmarks show competitive performance in general intelligence, coding, and agentic tasks, with particular strength in sustained coding work supported by its one-million-token context window.
