News for hackers. Stuff that matters.
Re:ChatGPT, Rot13, and Daniel Kahneman by pa7x1 on Friday December 09, 2022 @04:07AM (#33919248)
This is not it or not sufficient explanation. chatGPT is able to read text with letters purposefully scrambled. Like those texts that show how humans have an innate ability to read text with letters misplaced. chatGPT can understand the text, figure out what is going on and produce text of the same type. I tried it, precisely to understand if it was working with words as its token.

Interestingly, it cannot produce text with the order or characters reversed or guess that's what is going on. And if you explain it, it's able to produce code that would reverse the characters but is still not able to apply it for long sentences.

Re:ChatGPT, Rot13, and Daniel Kahneman by nverno on Friday December 09, 2022 @05:25AM (#33919762)
Did you try entering one of those scrambled sentences into the tokenizer? It's not tokenizing words exactly- the doc says roughly ~100 tokens => 75 words.
Re:ChatGPT, Rot13, and Daniel Kahneman by pa7x1 on Friday December 09, 2022 @09:33AM (#33921572)
I just tried out of curiosity. The jumbled words get broken up into different tokens, the unjumbled words do not get split.

https://imgur.com/a/a3zmkIv