Researchers introduced Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization (SA-MRPO), a method for multi-objective reinforcement learning that normalizes each reward objective independently and adaptively discounts its contribution based on a batch-level estimate of how saturated that objective already is. This reallocates optimization effort toward under-optimized objectives rather than using fixed weights throughout training. Across mathematical reasoning, adaptive reasoning, and code-generation tasks, SA-MRPO improved performance on harder objectives, including gains of up to 5% on AIME24, while preserving performance on objectives already satisfied, and the method reduces to existing approaches like GDPO and GRPO as special cases.
