News for hackers. Stuff that matters.

A comment on tiktoken: OpenAI’s Tokenizer. Parent comment

Re:tiktoken: OpenAI’s Tokenizer by TechBro8615 on Friday December 16, 2022 @02:55PM (#34019406)
I see "tiktok en" like en.wikipedia.org or microsoft.com/en
[ Parent | on HN ]

There are no comments on this comment yet.