News for hackers. Stuff that matters.
Re:OpenAI Tokenizer by hombre_fatal on Wednesday April 05, 2023 @11:36AM (#35455941)
Of course, you can find BPE tokenizer impls in other languages for the same results as long as they use the "cl100k_base" data.

Typescript: https://github.com/danneu/telegram-chatgpt-bot/tree/master/t...

There are no comments on this comment yet.