I want the indexer token-selection helpers to expose pure functions for turning document fields into the…

I want the indexer token-selection helpers to expose pure functions for turning document fields into the…: a task in MiMo-V2.6-RL-harbor-code: MiMo-V2.6-RL Code (Harbor) (Harbor dataset). prepare url for tokenizing(url: str) - str should strip a leading http:// or https:// and replace / , . , and…