Introduce a ParallelismConfig dataclass that composes DDP / FSDP / HSDP / TP / CP into a single named torch…
Introduce a ParallelismConfig dataclass that composes DDP / FSDP / HSDP / TP / CP into a single named torch…: a task in HF ML Bench v0 (Harbor dataset). Design and implement a first-class configuration object in accelerate that captures a modern N-D parallelism plan for a training run. Users must…
The task
Design and implement a first-class configuration object in `accelerate` that captures a modern N-D parallelism plan for a training run. Users must be able to declare, in a single dataclass, how many ranks to devote to each of:
Part of AdithyaSK/HF_ML_Bench_v0.