HF RL Explorer

Add support for periodic reference model synchronization (TR-DPO style) to KTOConfig and KTOTrainer …

Add support for periodic reference model synchronization (TR-DPO style) to KTOConfig and KTOTrainer …: a task in HF ML Tasksmith (Harbor dataset). When sync ref model=True , the reference model is periodically updated: π ref = α · π θ + (1−α) · π ref prev , where α is ref model mixup alpha and the…

Part of FineEnvs/HF_ML_Tasksmith.