OK, I can partly explain the LLM chess weirdness now

Re:OK, I can partly explain the LLM chess weirdness now by david-gpu on Friday November 22, 2024 @05:49AM (#42212744)

> in appropriate situations, humans are hugely more data-efficient

After spending some years raising my children I gave up the notion that humans are data efficient. It takes a mind numbing amount of training to get them to learn the most basic skills.

[ Parent | on HN ]

Re:OK, I can partly explain the LLM chess weirdness now by rscho on Friday November 22, 2024 @07:57AM (#42213448)

You could compare childhood with the training phase of a model. Still think humans are not data-efficient ?

[ Parent | on HN ]

Re:OK, I can partly explain the LLM chess weirdness now by david-gpu on Friday November 22, 2024 @08:03AM (#42213493)

Yes, that is exactly the point I am making. It takes many repetitions (epochs) to teach them anything.

[ Parent | on HN ]

Re:OK, I can partly explain the LLM chess weirdness now by rscho on Friday November 22, 2024 @08:07AM (#42213523)

Compared to the amount of data needed to train an even remotely impressive 'AI' model , that is not even AGI and hallucinates on a regular basis ? On the contrary, it seems to me that humans and their children are hugely efficient.

[ Parent | on HN ]

Re:OK, I can partly explain the LLM chess weirdness now by david-gpu on Friday November 22, 2024 @01:09PM (#42216057)

> On the contrary, it seems to me that humans and their children are hugely efficient.

Does a child remotely know as much as ChatGPT? Is it able to reason remotely as well?

[ Parent | on HN ]

Re:OK, I can partly explain the LLM chess weirdness now by rscho on Friday November 22, 2024 @01:56PM (#42216439)

I'd say the kid knows more about the world than ChatGPT, yes. For starters, the kid has representations of concepts such as 'blue color' because eyes... ChatGPT can answer difficult questions for sure, but overall I'd say it's much more specialized and limited than a kid. However, I also think that's mostly comparing apples and oranges, and that one's judgement about that is very personal. So, in the end I don't know.

[ Parent | on HN ]

Re:OK, I can partly explain the LLM chess weirdness now by meroes on Friday November 22, 2024 @09:16PM (#42218765)

A baby learns to walk and talk in 1 year. Compared to the number of PHDs and compute training these models, the baby is so far ahead in efficiency I marvel way more at their pace.

[ Parent | on HN ]