News for hackers. Stuff that matters.

A comment on Generative AI uses more energy on non-English text. Parent comment

Re:Generative AI uses more energy on non-English text by ben_w on Saturday November 14, 2026 @02:17PM (#50499889)
A token is an index into a dictionary. What the dictionary uses isn't seen, this is why they couldn't count the 'r's in 'strawberry'.
[ Parent | on HN ]
Re:Generative AI uses more energy on non-English text by paulryanrogers on Saturday November 14, 2026 @08:55PM (#50503229)
I was thinking both of input and latent data. Hadn't thought of tokens that way. Thanks for the heads up.
[ Parent | on HN ]
Re:Generative AI uses more energy on non-English text by stephantul on Sunday November 15, 2026 @04:47AM (#50505502)
H net directly operates on bytes. This is what the comment was about and what I was referring to.
[ Parent | on HN ]