HF RL Explorer

I want the indexer token-selection helpers to expose pure functions for turning document fields into the…

I want the indexer token-selection helpers to expose pure functions for turning document fields into the…: a task in MiMo-V2.6-RL-oss: Agentic RL Environments (MiMo RL release). The main entry point should be tokenize document(url: str, title cleaned: str, extract: str, score) - TokenizedDocument…

The task

The main entry point should be tokenize_document(url: str, title_cleaned: str, extract: str, score) -> TokenizedDocument, and it should return a TokenizedDocument with the original url, title, extract, and score values plus a token list built from the title, prepared URL, and…

Part of XiaomiMiMo/MiMo-V2.6-RL-oss.