DaoCloud deployed GLM-5.2-NVFP4, a 744-billion-parameter mixture-of-experts model, across 24 NVIDIA B300 GPUs using a disaggregated prefill-decode topology, achieving mean token-per-output-time of 17 milliseconds against an SLA target of 20 milliseconds. The team reduced initial latency from approximately 40 milliseconds through speculative padding optimization, Model Runner V2 improvements, and all-to-all communication backends. Production stability testing revealed a previously undetected host-memory leak in block allocation tracking that affected models without Mamba layers, subsequently fixed in the upstream vLLM repository.
