Dion3 targets the computational bottleneck of orthogonalizing weight updates in the Muon optimizer, introducing a Gram Newton-Schulz algorithm, specialized kernel acceleration, and a megabatching communication strategy, plus a technique that selectively orthogonalizes only a subset of momentum-matrix rows per step. The combined changes deliver up to a 6x speedup in optimizer step time compared to the earlier Dion variant, while matching or exceeding Muon’s loss performance; the implementation is released as an open-source package.
