HF RL Explorer

Duplicate-file finder dedup.py: walk the given directories, group candidates by size first, then by sha256…

Duplicate-file finder dedup.py: walk the given directories, group candidates by size first, then by sha256…: a task in traj2envs-rl: RL tasks generated with traj2envs (Harbor dataset). Consequence: files that differ in size, or whose first 4 KiB differ, are never fully hashed.

The task

Consequence: files that differ in size, or whose first 4 KiB differ, are never fully hashed.

Part of harness-hub/traj2envs-rl.