The authors present CRISP, a method for speeding up the attention prefilling phase of long-context LLM inference. It replaces a Jensen-Shannon-Divergence routing mechanism with a lighter structural proxy that measures attention mass at Vertical-Slash-compatible positions, and introduces a sink-aware thresholding approach to address a post-softmax mass cliff problem that otherwise lets accumulated background noise corrupt token selection. On InfiniteBench, RULER, and LongBench, CRISP achieves up to 5.30x attention speedup at 512k-token context while matching dense-attention accuracy on retrieval-heavy tasks.
