Character detection fails for Cyrillic text in file names
Character detection fails for Cyrillic text in file names: a task in harbor-swesmith-rl-artifacts: Harbor SWE-Smith 强化学习数据产物 (Harbor dataset). I've discovered an issue with the character detection system in charset normalizer. When processing file names containing Cyrillic characters, the system…
The task
I've discovered an issue with the character detection system in charset_normalizer. When processing file names containing Cyrillic characters, the system fails to properly identify them as Cyrillic text.