News for hackers. Stuff that matters.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by leobg on Tuesday April 11, 2023 @02:42AM (#35521651)
GPT is auto regressive. That means each output token becomes part of the new input sequence. Which is to say, the beginning of the model’s answer becomes part of your prompt.

If the model makes some mistake in the beginning, it now needs to explain / make sense of that mistake.

Kind of like a split-brain patient whom you ask why they got up, and they then say, to get a Coke. [1] In psychology, that is called confabulation. In machine learning, they use “hallucination“, probably so they can use the term across several disciplines, like language, audio, vision, etc.

[1] https://www.brainscape.com/flashcards/chapter-4-hemispheric-...

Re:Teaching ChatGPT to Speak My Son’s Invented Language by Jiocus on Tuesday April 11, 2023 @10:43AM (#35525873)
The psychology of split-brain is a nice gateway to 'The Bicameral Mind', a major theme about how the conception of the AI's works in the series Westworld. Nice!
Re:Teaching ChatGPT to Speak My Son’s Invented Language by leobg on Tuesday April 11, 2023 @02:35PM (#35529310)
Thank you. I am, in fact, reading it. Fascinating read. (How come all the best books were written in the 70s? Just survivership bias?)
Re:Teaching ChatGPT to Speak My Son’s Invented Language by jychang on Wednesday April 12, 2023 @04:58AM (#35536768)
Because the 1970s had bad science that was poorly replicated, especially in psychology.

Just because a theory sounds nice and seems to make sense doesn’t mean it’s scientific