Researchers introduce Grouped Value Attention (GVA), a Transformer attention variant that stores grouped values while reconstructing content keys via a learned linear map absorbed into the query at inference time, eliminating the need to materialize keys in the decode path. A shared decoupled RoPE channel preserves positional information through a separately cached positional key. At the 350M-parameter scale trained on 30B FineWeb-Edu tokens, GVA cuts persistent KV-cache scalars by roughly 45-47% relative to matched grouped-query attention while reaching comparable average accuracy (44.18 vs. 44.36 for GQA) across five benchmark tasks. The authors have built custom decoding kernels and plan an open-source release to translate the compact cache into faster autoregressive inference.
