> thinking (ALL the thinking) on par with a highly trained human
you are mistaking means for ends
Some like Penrose even argue that the global optimum of general intelligence and consciousness is a fully physical process, yes, but that involves uncomputable physics and thus permanently out of reach of whatever computers can do.
And yet is somehow within reach of a fertilised human egg.
It's time to either invoke mystical dimensions of reality separating us from those barbarian computers, or admit that one day soon they'll be able to do intelligence too.
Penrose's argument is interesting and I am inclined to agree with it. I might very well be wrong, but I don't think the accusation of magical thinking is warranted.
This is wrong. Computability is by no means the same as physicality. That's the whole point and you're just ignoring it to make some strawman accusation of ridiculousness.
2. If that property is a physical property, what prevents simulation of it?
The question is malformed. How can something compute an uncomputable thing?
> 2. If that property is a physical property, what prevents simulation of it?
Is P=NP?
If humans cannot compute uncomputable things either, on what grounds do you claim we are capable of something that computers are incapable of?
I know you think this is a gotcha moment so I will just sing off on this note. You think physical = computable. I think physical > computable. I understand your argument and disagree with it but you can't seem to understand mine.
I've tried to follow your reasoning, which AFAICT comes down to a claim that humans possess something connected to incomputability, and computers do not. But now it seems you hold this difference to be irrelevant.
So again: What do you think the difference in capability between humans and computers is?
Understimulated or feral children don’t automatically become geniuses when given more information.
It takes social engineering and tons of accumulated knowledge over the lifespan of the maturation of these eggs. The social and informational knowledge are then also informed by these individuals (how to work and cooperate with each other, building and discovering knowledge beyond what a single fertilized egg is able to do).
This isn’t simply within reach of a fertilized egg based on its biological properties.
Current LLMs seem to be most similar to the linguistic/auditory portion of our cognitive system, and with thinking/reasoning models some bits of the executive function. But my guess is that if we want to see awe-inspiring stuff come out of them, we need stuff like motivation and emotion, which doesn't seem to be the direction we're heading towards.
Unprofitable, full of problems. Maybe 1 in 100,000 might be an awe-inspiring genius, given the right training, environment, and other intelligences (so you might have to train way more than 100K models).
This is the easiest part. Keyboards are only necessary if you have fingers, an AI could very easily send midi notes directly to the instrument.
Is it? I wasn’t aware of “playing a physical instrument on stage with millisecond response times” as a criterion. I’m also confused by the implication that professional composers aren’t using intelligence in their work.
You’re talking about what is sometimes called “superhuman AGI”, human level performance in all things. But AGI includes reaching human levels of performance across a range of cognitive tasks, not ALL cognitive tasks.
If someone claimed they had invented AGI because amongst other things, it could churn out a fresh, original, good composition the day after hearing new input - I think it would be fair to argue that is human level performance in composition.
Defining fresh, good, original is what makes it composition. Not whether it was done in real time; that’s just mechanics.
You can conceivably build something that plays live on stage, responding to other players, creating a “new work”, using super fast detection and probabilistic functions without any intelligence at all.
This has aged well and continues to do so.
Can we make a program that could: work the physical instrument, react in milliseconds to other players, create new work off input it's never heard before, react to things in the room, and do a myriad of other things a performer does on stage in real time? Not even remotely close. THAT is what would be required to call it AGI – thinking (ALL the thinking) on par with a highly trained human. Pretending anything else is AGI is nonsense.
Is it impressive? sure. Frighteningly so even. But it's not AGI and the claims that it is are pure huckersterism. They just round the term down to whatever the hell is convenient for the pitch.