News for hackers. Stuff that matters.

A comment on Generative AI uses more energy on non-English text.

Re:Generative AI uses more energy on non-English text by robrenaud on Saturday November 14, 2026 @10:05AM (#50497593)
Use different tokenizers for different languages. Or at least, use different tokenizers for different scripts.
[ Parent | on HN ]

There are no comments on this comment yet.