Feature Request: Configurable parallelism for complement data in parallel execution mode
Feature Request: Configurable parallelism for complement data in parallel execution mode: a task in LegoFlow-SWE (Harbor dataset). Problem When running complement data (also called supplementary/recording data) tasks using parallel execution mode, DolphinScheduler generates and submits all process…
The task
**Problem** When running complement data (also called supplementary/recording data) tasks using parallel execution mode, DolphinScheduler generates and submits **all** process instances simultaneously with no concurrency control. For long date ranges this can exhaust server memory and cause crashes.
Part of Lego-X/LegoFlow-SWE.