Proposes Posterior Label Correction DPO (PLC-DPO), which routes each preference pair as clean, flipped, or tied using the calibrated policy-reference margin as online evidence, rather than only filtering suspicious pairs. Reframes noisy preference learning as actively correcting the direction and strength of supervision signals during DPO training. Across 57 dataset-model-benchmark cells, PLC-DPO achieves a 60.5% mean win rate against standard DPO, ahead of the next-best method’s 55.5%, with stress tests confirming the routing distinguishes flipped from weakly directional pairs.
