The vLLM-Omni team detailed model-specific engineering work to optimize text-to-speech inference across four models. For Qwen3-TTS the team decoupled streaming chunks from decode windows and added CUDA Graph caching by frame count, while VoxCPM2 used whole-forward torch.compile and decode-tail batching across requests. Fish Speech S2 Pro received a custom Triton attention kernel for single-token decode shapes, and Higgs Audio V3 moved multi-codebook state management onto GPU-resident tensors with CUDA graphs adapted for dynamic batch sizes.
