HF RL Explorer

Feature Request: Implement AutoscaledPool concurrency manager and measure time utility

Feature Request: Implement AutoscaledPool concurrency manager and measure time utility: a task in LegoFlow-SWE (Harbor dataset). We need an async concurrency manager that automatically scales the number of parallel tasks based on system CPU and memory load. It should allow graceful shutdown via an…

The task

We need an async concurrency manager that automatically scales the number of parallel tasks based on system CPU and memory load. It should allow graceful shutdown via an abort mechanism and provide a context manager to measure execution time (wall and CPU). This component is inspired by the TypeScript Crawlee…

Part of Lego-X/LegoFlow-SWE.