Re:Transformers struggle with generalizing tasks beyond pre-...
by Isomorpheus on Tuesday November 07, 2023 @01:11PM
(#38180531)
Can you name three things an average child knows which GPT-4 doesn't?
Re:Transformers struggle with generalizing tasks beyond pre-...
by abeppu on Tuesday November 07, 2023 @03:34PM
(#38182435)
How to drink from a cup without dribbling.
The feeling of sunshine on one's skin.
The experience of the passage of time.
Re:Transformers struggle with generalizing tasks beyond pre-...
by tomohelix on Tuesday November 07, 2023 @04:12PM
(#38182917)
I understand what you are saying so I will address the core of your question instead of just giving you things a child can do.
An LLM right now is still just a facsimile of one of human's cognition. Arguably, without the other senses, it can't experience or understand things the same way we can. This leads to the AI unable to form a comparable sentience to a child, at least, not in the same way and not in a way that would be recognized by most people.
Someone already gave you the sense of time as an example. These senses are things we were born with and critical to our perception of the surroundings. Until the AI is trained on at least the majority of these senses, a child will always have something they can do that the AI can't.
LLMs are trained on a tiny subset of the written human knowledge which we've proven to probably not be garbage and which is nicely formatted in simple text formats and which was published without too many paywalls on the web and on and on and on. It's a lot, and it definitely includes enough facts that no one person knows all the things the LLM "knows", but the average child knows plenty of things which never made it into that sort of a corpus. Yes, it's probably true that the vast majority of humans are also not pushing the frontier of human knowledge forward, but the vast majority of humans are working with a slightly different (partially overlapping) set of information than what the LLMs see.