The paper identifies ‘phase sensitivity’ in transformers using chunked KV-cache compression for long-context inference: retrieval accuracy varies systematically (by up to 40 percentage points) depending on a token’s position relative to compression-window boundaries, a pattern hidden by averaged benchmark scores. Through pretraining multiple transformer variants and mechanistic analysis, the authors trace this to phase-specialized attention heads and argue gradient-flow dynamics drive the effect in simplified retrieval models. They conclude that evaluating chunked KV-cache compressed models requires phase-specific measurement rather than aggregate accuracy.
