vLLM has released production-ready support for Kimi K3, a 2.8-trillion-parameter multimodal mixture-of-experts model featuring a hybrid architecture combining Kimi Delta Attention layers with periodic full-attention mechanisms. The implementation includes specialized optimizations such as a hybrid KV-cache manager for managing recurrent and paged KV blocks simultaneously, fused CUDA kernels for attention residuals and KDA operations, and support for DSpark speculative decoding that achieves approximately 3.14× throughput improvements. The deployment supports advanced features including prefix caching across hybrid cache types, prefill/decode disaggregation for large-scale serving, and intelligent cache retention policies designed for agentic workloads spanning million-token contexts.
