News for hackers. Stuff that matters.
Re:GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text by Arson9416 on Sunday December 03, 2023 @08:44AM (#38507001)
>trained to fix typos

It is trained on data which may include typos, but that is very different from fixing typos. It knows what words likely come after typos in the same way it knows what words likely come after regular words.

Re:GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text by lrei on Sunday December 03, 2023 @02:19PM (#38509858)
No, that's not what I meant. I meant that in its reinforcement learning phase, GPT saw examples of "fix this text" style requests and was rewarded for doing a good job. That's different from seeing examples of typos and still predicting the right word which happens during the language model self supervised training. Both likely help it be good at it.