SimpleOPD is a new on-policy distillation method for transferring long-context reasoning ability from large teacher models to smaller, shorter-context student models, addressing tokenizer mismatches between the two by operating in shared text space. The method adds a student-reference KL loss to prevent runaway generation length and training instability, and was tested across the Qwen3, Qwen3.5, and Intern-S2 model families. On mathematical reasoning benchmarks such as ProofBench, Intern-S2-Preview trained with SimpleOPD gained 21.2 points and surpassed Gemini-2.5-Pro.
