The paper introduces Minimal Intervention Reinforcement Learning (MInTRL), which expands the exploration frontier of on-policy RL with verifiable rewards by having a judge-intervention policy periodically review a rollout in progress, replace erroneous suffixes with short corrections, and immediately return control to the base policy. Training uses a sequence-level advantage-regression objective that avoids the need for importance sampling despite these local off-policy interventions. Across math and code benchmarks, MInTRL consistently outperforms both standard on-policy and off-policy baselines, with ablations showing it remains effective under self-intervention and across different judge policies, and that performance peaks at moderate rather than heavy intervention.
