Repair Dask's two Parquet metadata engines for all-or-none overlap between Hive partitions and physical…
Repair Dask's two Parquet metadata engines for all-or-none overlap between Hive partitions and physical…: a task in MiMo-V2.6-RL-oss: Agentic RL Environments (MiMo RL release). Repair redundant Parquet partition metadata A local data pipeline reads Hive-style partitioned Parquet datasets through…
The task
Repair redundant Parquet partition metadata A local data pipeline reads Hive-style partitioned Parquet datasets through two Dask backends. Some producers retain every partition key as a physical file column, while others omit every key and rely on directory metadata. The…
Part of XiaomiMiMo/MiMo-V2.6-RL-oss.