Cross-lingual Ranking Preference Optimization (CRPO) transfers preference knowledge from English to improve LLM alignment in other languages, using a hierarchical structure within parallel preference pairs across a target language and English to jointly optimize intra- and inter-lingual preferences. Building on the LambdaRank loss, CRPO enables relative ranking comparisons among multiple candidate responses rather than simple binary preference judgments.
