HF RL Explorer

Implement tensor parallelism for linear layers using PyTorch.

Implement tensor parallelism for linear layers using PyTorch.: a task in Task Alignment Bench (TAB) (Harbor dataset). You will be able to fetch the world size and rank of the current process using torch.distributed.get world size() and torch.distributed.get rank().

The task

You will be able to fetch the world_size and rank of the current process using torch.distributed.get_world_size() and torch.distributed.get_rank().

Part of anonymous98570239854/tab.