This paper introduces triadic linear attention, which extends linear-attention mechanisms by replacing matrix-valued recurrent states with three-dimensional tensor states. The method writes the outer product of a key, a second key, and a value into a tensor and reads from it via dual query contractions, so an E-dimensional second key yields an E-fold state expansion using only two extra projections. It integrates with existing techniques such as data-dependent forgetting and chunkwise-parallel training. Experiments show it improves long-context language modeling and recall over alternative state-enlargement approaches.
