Scheduler is spending most of its time in clear not launched queued tasks function
Scheduler is spending most of its time in clear not launched queued tasks function: a task in MiMo-V2.6-RL-oss: Agentic RL Environments (MiMo RL release). Apache Airflow version 2.7.1 What happened Airflow running the clear not launched queued tasks function on a certain frequency (default 30…
The task
Apache Airflow version 2.7.1 What happened Airflow running the clear_not_launched_queued_tasks function on a certain frequency (default 30 seconds). When we run the airflow on a large Kube cluster (pods more than > 5K). Internally the clear_not_launched_queued_tasks function…
Part of XiaomiMiMo/MiMo-V2.6-RL-oss.