Re:Weird GPT-4 behavior for the specific string “ davidjl”
by godelski on Saturday June 10, 2023 @08:46PM
(#36276885)
It's not a myth, it is a hallucination that works in English. Which used because we're all speaking that language here. Like I said above, it is a different bias, but we have different biases than LLMs. Every language is going to have different biases. Some classes of heteronyms don't even exist in certain languages so a similar thing to The Chaos doesn't work as well there either. Maybe Chomsky would say there are more clear examples since he believes in a universal grammar but I'm also not a linguist. Just wanted an example. I wouldn't call this cherry picking as it both works in a few latin languages as well as strongly works in English. But even in English there is a bit of care you need, like double letters help to be stuck together. That's not cherry picking though as the example isn't such a unique thing that it invalidates the main thesis (what cherry picking is about).
[ Parent
| on HN ]
Re:Weird GPT-4 behavior for the specific string “ davidjl”
by Sharlin on Sunday June 11, 2023 @09:53AM
(#36281373)
It’s not a hallucination. It’s simply error correction, just like optical disks or ECC RAM are resistant to scrambled bits thanks to redundant, low-entropy coding – the fact that the bits are correlated.