Separator DSL methods produce incorrect lookahead / possible tokens
Separator DSL methods produce incorrect lookahead / possible tokens: a task in MiMo-V2.6-RL-oss: Agentic RL Environments (MiMo RL release). When a rule uses the separator variants of the repetition DSL methods (MANY SEP / AT LEAST ONE SEP), the computation of the possible next tokens / lookahead…
The task
When a rule uses the separator variants of the repetition DSL methods (MANY_SEP / AT_LEAST_ONE_SEP), the computation of the possible next tokens / lookahead paths gives wrong results. For example, given a rule along the lines of: After consuming an Identifier, the set of…
Part of XiaomiMiMo/MiMo-V2.6-RL-oss.