Check out our new preprint on cross-lingual transfer of reward models!