News for hackers. Stuff that matters.
Re:Generative AI uses more energy on non-English text by akssri on Saturday November 14, 2026 @10:18PM (#50503658)
> When working in other languages, the models used more energy because they had to generate more tokens...

Seems a bit dated. Sarvam has been taking about the inefficiency of the tokenization schemes used by mainstream models when it comes to Indic-languages for years now.

Re:Generative AI uses more energy on non-English text by kgeist on Saturday November 14, 2026 @10:34PM (#50503741)
There's Russian research on this, they swapped the tokenizer of an existing model (IIRC something from the Qwen family) for one which had far more Cyrillic tokens. After some post-training, the model kept its original quality, but it used way fewer tokens.