Feature request: add hardsilu / hardswish activation
Feature request: add hardsilu / hardswish activation: a task in MiMo-V2.6-RL-oss: Agentic RL Environments (MiMo RL release). I was looking through ivy's experimental activations and noticed quite a lot of them are already there — silu, relu6, hardtanh, hardshrink, selu, elu, celu, etc. — but I…
The task
I was looking through ivy's experimental activations and noticed quite a lot of them are already there — silu, relu6, hardtanh, hardshrink, selu, elu, celu, etc. — but I couldn't find hardsilu (a.k.a. hardswish). It's a pretty standard activation at this point (MobileNetV3…
Part of XiaomiMiMo/MiMo-V2.6-RL-oss.