News for hackers. Stuff that matters.

A comment on Generative AI uses more energy on non-English text. Parent comment

Re:Generative AI uses more energy on non-English text by cubefox on Saturday November 14, 2026 @12:55PM (#50499159)
They are typically optimized for English, for example, they often use rules for English contractions. See this thread: https://x.com/_albertgu/status/2113687798826111283

I don't know whether European languages work inherently better with tokenizers, but I don't see why they would.

[ Parent | on HN ]

There are no comments on this comment yet.