The paper proposes BeaconKV, a training-free technique that compresses key-value caches during inference for large reasoning models. It identifies that certain decoding steps revisit distant earlier context, a pattern existing compression methods miss, and maintains compact ‘beacon query’ representatives to anticipate which cache entries will be needed. The method delivers up to 5.8x memory reduction while nearly preserving full-cache accuracy and improving throughput by more than 4.3x, making extended reasoning tasks more feasible on resource-constrained hardware.
