Proposes Self-Retiring On-Policy Distillation, where a skill-conditioned teacher trains jointly with RL and the student adaptively drops the teacher once the performance gap stops shrinking, improving ALFWorld success by 14.1-18.8% and WebShop accuracy by 11.8-19.0% over RL baselines across Qwen2.5 1.5B-7B.