Re:Why is GPT-3 15.77x more expensive for certain languages?
by k8si on Monday April 10, 2023 @03:33PM
(#35516387)
I don't know what you mean by compiler terms but basically, worse tokenizer = worse LM performance. This is because worse tokenizer means more tokens per sentence so it takes more FLOPs to train on each sentence, on average. So given a fixed training budget, English essentially gets more "learning per token" than other languages.
[ Parent
| on HN ]
There are no comments on this comment yet.