Researchers introduce RestoreKV, a technique that addresses the accuracy loss large language models suffer under aggressive key-value cache eviction by pairing selection-based eviction with a complementary restoration mechanism. The method generates a compact, context-conditioned restore cache using eight learnable tokens with LoRA-adapted attention, trained via self-distillation from full-cache models while updating only about 0.4% of model parameters. On the RULER-4K benchmark at a 5% cache budget, RestoreKV raises KVzip accuracy from 38.2 to 73.2 while adding less than 0.5% additional construction time, improving results across multiple benchmarks and eviction methods with negligible overhead.
