News for hackers. Stuff that matters.
Re:GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text by sp332 on Sunday December 03, 2023 @03:13PM (#38510335)
If you put it into the tokenizer https://platform.openai.com/tokenizer you can see that it helps in some places but not in others. It pulled out "SEA/OF/TRAN/QU/ILITY", but I think it broke up every instance of the word "THE".

There are no comments on this comment yet.