The paper analyzes gradient estimation noise in sparse on-policy distillation and proposes an ‘information-efficiency ratio’ (IER) metric, based on signal-to-noise decomposition, to characterize gradient error when teacher guidance is sampled from student-generated trajectories. Combining IER with existing token-selection methods enables sparse supervision at only 0.1%-1% of tokens while matching or exceeding full distillation performance on math and medical reasoning tasks.
