Duplicate-file finder dedup.py: walk the given directories, group candidates by size first, then by sha256…
Duplicate-file finder dedup.py: walk the given directories, group candidates by size first, then by sha256…: a task in traj2envs-rl: RL tasks generated with traj2envs (Harbor dataset). Consequence: files that differ in size, or whose first 4 KiB differ, are never fully hashed.
The task
Consequence: files that differ in size, or whose first 4 KiB differ, are never fully hashed.
Part of harness-hub/traj2envs-rl.