SpectralShift: Effective Context Window Extension of Gated DeltaNet via Spectral Reparameterization
The paper studies long-context extension of Gated DeltaNet, a linear-attention architecture, through the spectral properties of its state transition matrix…