Re:Why is GPT-3 15.77x more expensive for certain languages?
by jltsiren on Monday April 10, 2023 @03:52PM
(#35516614)
It's more like some big languages receive special treatment, while everything else is interpreted as a byte stream. In Finnish language, the tokens seem to be arbitrary substrings of average length 3-4, and they rarely correspond to any semantically or grammatically meaningful units.
[ Parent
| on HN ]
There are no comments on this comment yet.