I mean, sure, I can totally see the idea that consciousness is maybe some emergent property of complex systems... in which case, sure, we could stumble upon it by accident. That is totally possible; we're creating complex self-replicating systems all over the place.
But as far as intentionally constructing the thing that will come after humanity? we can't do that until we know what, exactly, it is we are constructing, and as far as I can tell, we're pretty far away from any idea of exactly what that is.
I imagine this is what's going to happen: we humans have drawn two circles (thank you dear fella for that analogy) and we are about to draw a third, combing two existing concepts.
At some point we will have drawn so many circles that the next one will not be drawn by us. It will be drawn by AI.
Who knows when we'll reach that level of machine intelligence and if it even requires them, the machines, to be conscious? But we will, without any doubt, reach a point where there have been so many circles drawn that they are now smarter than us at drawing them. Absolutely without any sort of doubt in my mind this will happen.
Tens of thousands of people comprehend Einstein's work. It used to be hundreds. We're smart as shit these days. Fuck me if it's not before I die.
In some ways, I feel like this is the real singularity. This trend towards feeding and offering education to everyone. Like, it used to be that to be an intellectual, you'd need to be born rich to get the free time and education. I mean, being born rich still helps, sure, but a lot more people are getting a shot.
On the other hand, it's a process that levels off at some point, like everything else. The population is leveling off; as we bring education and leisure time to more and more of the world, eventually most of the people who have the capability to do this sort of thing will have done it.
I mean, I see that with computers, too... like with the leveling out of Moore's law; exponential processes, in nature, tend to not remain exponential.
I always thought so. Like, without volition, it's just a really good natural language tool, and not a general intelligence.
What would an artificial general intelligence without consciousness look like to you?
IMO this is much more plausible as AGI than some kind of mythical general meta-intelligence that has a virtual soul.
I don't see why consciousness or agency are at all necessary for AGI - except as a science fiction trope.
so, like a philosophical zombie? like something that appears to be conscious but isn't?
So basically if we can have a system that can solve any problem thrown at it, that's essentially the AGI.
Capitalistic dream.
If such beings were conscious they would be saints.
One of my favorite AI jokes:
Q: What's AI?
A: When the machine wakes up and says, "What's in it for me?"
It would be fetish to actually make humanoid servitor robots. IMO.
Is creativity possible without consciousness? I don't pretend to be an expert.
The idea was that such an AI would never question its priorities or change them, not that it wouldn't be able to solve novel problems.
(Which is the definition of intelligence, duh. And shows how far we're from that. What we have is glorified pattern matching algorithms and some basic symbolic logic and clustering.)