PyTorch 2.13 was released on July 8, 2026, bringing FlexAttention support to Apple Silicon (MPS) with up to 12x speedups on sparse attention patterns and a new CuTeDSL backend for Inductor that generates CUTLASS-grade GEMM kernels with faster compilation. The release adds nn.LinearCrossEntropyLoss, which cuts peak GPU memory by up to 4x during large-vocabulary language model training, along with a new torchcomms backend for more resilient distributed training. Additional improvements include FSDP2 support for overlapping all-gather and reduce-scatter communications via separate process groups, plus Python 3.15 wheels with experimental free-threaded builds and platform enhancements for ROCm, Arm, and Intel XPU.
