The paper identifies that FP8 quantization noise in RL pipelines distorts importance ratios, causing training instability. It proposes ‘Calibrated Clipping,’ which aligns FP8 clipping bounds with high-precision BF16 distributions by matching the lower-bound clipping quantile and rebalancing the upper bound, eliminating training instabilities across multiple RL algorithms and model sizes.
