News for hackers. Stuff that matters.
Re:OpenAI Tokenizer by teruakohatu on Wednesday April 05, 2023 @05:53PM (#35460896)
That is very helpful, thank you. I had not realised the latest models were now tokenizing number as 3 digit groups. Can you give any insight into why 3 digits?

There are no comments on this comment yet.