News for hackers. Stuff that matters.

A comment on Understanding GPT Tokenizers. Parent comment

Re:Understanding GPT Tokenizers by josecyc on Thursday June 22, 2023 @10:30PM (#36442110)
Where are all the tokens for other languages though? How are these getting tokenized?
[ Parent | on HN ]

There are no comments on this comment yet.