Prefix Sliding is an inference technique that discards reasoning tokens outside of a fixed prefix (containing key instructions and tools) and a sliding window of the most recent tokens, based on the finding that most intermediate reasoning tokens quickly lose importance as a model keeps reasoning. This caps total memory use regardless of how long a model reasons, making long test-time scaling tractable, and the authors report it makes existing models up to 3x faster without any retraining while maintaining performance. Training a model with Prefix Sliding via reinforcement learning further improves results by enabling reasoning traces beyond 100,000 tokens, and the authors have released the code.
