Fix double-backprop through shared feature networks
Fix double-backprop through shared feature networks: a task in MiMo-V2.6-RL-harbor-code: MiMo-V2.6-RL Code (Harbor) (Harbor dataset). In our actor-critic setup, a single FeatureNetwork produces a feature representation that is shared by several downstream heads (e.g. a value function and a…
Part of FineEnvs/MiMo-V2.6-RL-harbor-code.