Sliding window inconsistency between PyTorch and Flax
Sliding window inconsistency between PyTorch and Flax: a task in MiMo-V2.6-RL-oss: Agentic RL Environments (MiMo RL release). System Info transformers main (ae49b218c), Python 3.10.8 Who can help? @ArthurZucker, @sanchit-gandhi Reproduction The attention sliding window has different interpretation…
The task
System Info transformers main (ae49b218c), Python 3.10.8 Who can help? @ArthurZucker, @sanchit-gandhi Reproduction The attention sliding_window has different interpretation for PyTorch and Flax. Here's are matching examples: PyTorch Flax Both snippets return different results…
Part of XiaomiMiMo/MiMo-V2.6-RL-oss.