Many of the answers ChatGPT gives would also what a human with average knowledge would give if you point a gun to their head and demand an answer.

My gut feeling is that global long term memory and what we call hallucinations right now might actually be the next step to get closer to general intelligence.

I’m not sure we have the right reinforcement models yet tho, as counter intuitive as it might sound I don’t think that a reinforcement model that is based on correctness only is what we need.

Humans make mistakes all the time, and we bullshit all the time. If anything I think that ChatGPT scares people not because it gets things right but because how confidently it gets things wrong which is something we do all the time.