Hugging Face detailed improvements to the transformers library’s vLLM backend that let it match the throughput of hand-written native vLLM implementations. The backend now uses torch.fx for static graph analysis and AST manipulation to apply runtime layer fusions such as MergedColumnParallelLinear and QKVParallelLinear kernels. Benchmarks across Qwen3 models ranging from 4B to 235B parameters, under different parallelism strategies, showed comparable throughput to hand-optimized implementations, letting model authors get fast inference without writing custom optimization code.
