TokenSpeed-Kernel is an open-source subsystem that simplifies LLM inference backend complexity through a layered API and registry system separating high-level runtime operations from hardware-specific kernel implementations. Platform-agnostic APIs such as mha_prefill and moe_apply dispatch to optimized kernels registered for specific hardware targets, letting AMD and NVIDIA implementations coexist as sibling backends without device-specific logic embedded in model code. Using specialized Gluon kernels, GPT-OSS 120B on AMD MI355X achieves 1.6-3.6x throughput improvements over portable Triton baselines, and the reusable kernel packages can be adopted by other inference engines like vLLM independent of the full TokenSpeed serving stack.
