Re:Cerebras-GPT vs. LLaMA AI Model Performance Comparison
by sebzim4500 on Thursday March 30, 2023 @07:28AM
(#35371031)
>These are not the same things.
You mean because the token ids are in a different order? This is irrelevant to a transformer model, there is no inductive bias that similar tokens should have related token ids.
>Interestingly, the code FB provided doesn't seem to have any special handling for digits
You mean because the token ids are in a different order? This is irrelevant to a transformer model, there is no inductive bias that similar tokens should have related token ids.
>Interestingly, the code FB provided doesn't seem to have any special handling for digits
Yeah, token handling is done in sentencepiece.