Add a first-class conversion path in PEFT that distills non-LoRA adapters (LoKr, LoHa, and similar) into…
Add a first-class conversion path in PEFT that distills non-LoRA adapters (LoKr, LoHa, and similar) into…: a task in HF ML Bench v0 (Harbor dataset). Extend PEFT with a first-class conversion path so that many non-LoRA adapters (e.g. LoKr, LoHa, and other structured low-rank re-parameterizations)…
The task
Extend PEFT with a first-class conversion path so that many non-LoRA adapters (e.g. LoKr, LoHa, and other structured low-rank re-parameterizations) can be distilled into a plain LoRA adapter that reproduces the same effective delta weight on linear layers within reasonable numerical tolerance.
Part of AdithyaSK/HF_ML_Bench_v0.