HF RL Explorer

When using QLORA (Quantized LoRA) with FSDP (Fully Sharded Data Parallel) training, saving a model…

When using QLORA (Quantized LoRA) with FSDP (Fully Sharded Data Parallel) training, saving a model…: a task in LegoFlow-SWE (Harbor dataset). The error appears because the optimizer’s parameter groups contain parameters that cannot be correctly matched to the model’s parameters during the FSDP…

The task

The error appears because the optimizer’s parameter groups contain parameters that cannot be correctly matched to the model’s parameters during the FSDP optimizer state unflattening step. In a QLORA setup, frozen/non-trainable parameters (e.g., the quantized base model weights) are inadvertently included in the…

Part of Lego-X/LegoFlow-SWE.