Proposes SAS, a gated sparse attention mechanism that optimizes context-unit ranking end-to-end with the language modeling loss, rather than distilling dense attention distributions through non-differentiable Top-K selection. Uses log-form softmax gates and normalized calibration with continuous selector scores to better align ranking with prediction impact under a fixed attention budget. Validated on reasoning, long-context understanding, and agentic tasks, showing improvements over existing sparse-attention baselines.
