Quantum computing happening in the brain is an extremely extraordinary hypothesis, and requires some very good evidence before it's accepted. Quantum effects happening on the brain is a "well, duh, and you'll tell me water is wet later?" hypothesis, and requires some good evidence not to believe in it.
That's only one school of AI ("weak AI"). The other school ("strong AI") says that it's certainly worthwhile to create an intelligence that is capable of thought the way humans are -- if only to get a better handle on what "thought" and "intelligence" really are. Currently weak AI is "winning" because the surveillance and online-advertising industries can get more use out of it. But that doesn't put strong AI out of reach, nor render it wholly irrelevant.
My skepticism comes less from a hatred of "Strong AI" and more from the fact that it has been promised over and over again with no results to show for it. In addition, every time "weak AI" makes progress, there's always someone who writes an article bashing the progress of "weak AI" and saying "it isn't REAL AI (tm)".
This is kind of similar to the reason I dislike neuromorphic architectures, you can't just assume that you're right and look down upon all others, you need to show results if you want to do that.
Could that just be because Strong AI is much, much more difficult to achieve than Weak AI? And not that it is less valuable an end to pursue than Weak AI?
In other words, does the value of one vs. the other necessarily correlate to our previous success in one vs. the other?
But weak AI might be included in that.
How could that work concretely? I'm struggling to think of how an agent could be intelligent, e.g. able to effectively achieve its goals, without being aware of itself.
I suspect that, if you could communicate with one, any sufficiently intelligent being would be aware that it's a thinking being.
What else does "self-awareness" encompass that I'm missing?
The alien in Blindsight also apparently does not care to understand such things, and may lack the capacity for this kind of thought. It runs entirely on instinct, but these instincts bake in extremely complex behaviors that are beyond human capabilities.
I'm also reminded of two other aliens from other Sci Fi series: 1) the Orkz from Warhammer 40k. Dumb as a rock, but can build devastatingly effective vehicles and weapons from salvage and garbage. God knows how. They certainly have no idea. 2) The Motie engineers from "The Mote in God's Eye." They can instinctually build and improve on machines but cannot understand, or come close to explaining, How they did it.
The idiot-savant is not generally intelligent. Depending on what you mean by "complex mathematical problems" even the neural components actually responsible for solving the problems might not be intelligent. In the case of actual idiot-savants, I'd suspect those components aren't 'intelligent'; they're too simple.
The alien in Blindsight, if it really "runs entirely on instinct", then I'd expect it to do badly in novel situations, unless it's version of "instincts" basically "bake in extremely complex behaviors" that make it functionally equivalent or similar to a human – at the level we're used to dealing with as humans.
Also note that most of your examples are fictional, i.e. not actual evidence beyond maybe extremely weak plausibility. AFAIK, every "high intelligence" of which actual people have actual strong evidence is self aware. That seems like it should be an important fact!
Also, you're conflating self-awareness with the ability to (verbally) explain, i.e. communicate, thoughts to others. By "self-awareness", I was thinking of an 'intelligent system' including some sub-model of itself in whatever the relevant domain it is we're considering. For a 'general-purpose reproductive-fitness-maximizing-strategy executor' that's intelligent, e.g. us, it seems pretty unlikely that they don't, generally, model themselves.
We should probably taboo "self-awareness" as it's pretty obviously too slippery and ambiguous.
Well, a lot of very intelligent people never achieve their goals either.
I'd suggest that intelligence is not the achievement of goals but the formulation of goals. And funny enough, people generally have no self-awareness about how they come to have goals. Some kid grows up wanting to be a fighter pilot. Why? Why does some other kid want to be a writer?
Why do I like peach ice cream more than chocolate? Achieving the goal of getting some peach ice cream is a lot easier than understanding why I want to.
Intelligent beings want things, on their own. Rational consciousness is just the tool we use to get what we want. The importance of articulated self-awareness to intelligence is probably overstated by humans.
They're not intelligent with respect to the goals they're not achieving. Or they're competing with other people that are more 'intelligent'. There are (of course!) confounding factors but, all else being equal, the more intelligent agent should win.
Let's use a concrete example. Consider someone that's very good at solving math problems but is unhappy with their job. What's the problem with just considering that they're intelligent with respect to solving math problems but bad at (all of the skills necessary to be good at) finding a satisfying job?
Achieving goals, winning 'games', etc., seems like a really useful and really general way of measuring the effectiveness of things and that seems pretty close to, if not exactly, what intelligence is (if it's any one thing in particular).
> I'd suggest that intelligence is not the achievement of goals but the formulation of goals.
What's the motivation to define intelligence in that way?
Not necessarily. An intelligent agent could lose to an agent that can simply see the future.
Seeing the future has a tendency of becoming paradoxical when you start acting on that information though.
The paradoxes are the same as in time travel. Imagine your best sees itself hunted down two days in advance; it simply goes somewhere else so the hunters don't find it. Then it didn't really see the future, did it?
depends on your model of time and the universe, i suppose. perhaps "a future" vs "the future".
It's certainly relevant whether your AI technique involves creating conscious, suffering beings!
I think the more immediate concern is that we begin to turn over everyday life-and-death decisions (driving, obvs) to machines which are, at the end of the day, unconscious, and programmed by us oh-so-fallible human beings.
Exactly.
It's the distinction between programming consciousness and physically reproducing it. The latter is for physicists and biologists.
Equating abstractions with what it's abstracting is a mistake, but assuming something cannot be programmed or simulated is also a mistake. Anything observable can be abstracted.
But to then think something can be accurately simulated without a good understanding of its nature is another mistake. So programmers should be following physics and biology.
And finally, to equate the two when the simulation is accurate is yet another mistake.
It's like mistaking the gravity in a video game for real gravity. They are never the same. But they are related: One is a simulation of the other.
Biological systems work and evolved into intelligent self conscious beings without anyone who understands how they work guiding this evolution. Similarly self conscious AI can emerge without people having good understanding of the source of the consciousness.
Any evidence? It's fun to think about, but most people who think this are guilty of one of the mistakes I've listed. Or don't consider it a mistake. Or are not programmers.
AI, as long as it is software, is programmed with a very specific understanding of what is being programmed. Every line of code is intentional. Nothing exists in between the lines.
> guiding this evolution
AI is perfectly guided. Every program is completely understood (insert incompetent programmer joke here). Every symbol is typed with purpose on purpose.
The point is, Roger Penrose is not talking about programs. He's talking about physics.
When a programmer successfully emulates consciousness, every line will be deliberate, and they will know exactly how they did it. Not only that, if their program behaves like real consciousness, the programmer will have had to have knowledge of real consciousness. Otherwise what they've built would be something else. Something exciting I'm sure, but something else.
All of our models (and programs) are based on our understanding of the real world. Hence, they are simulations. The deeper our understanding of the real world, the more accurate our simulations.
I am not so sure. The current success of deep learning seems to be a direct counterexample to this. These techniques are yielding great success by imitating some patterns we observe in the brain, but the resulting inferences more or less defy rigorous explanation. We might fully understand the structure of a neural network but explaining the "why" behind its decisions seems to elude us.
> subjected to all kinds of experiences, which, in turn, will result in this brain becoming conscious - gradually.
Again, there is no evidence that "all kinds of experiences" will result in gradual emergence of consciousness from anything we have observed.
To a computer, experience is just data. So we're not talking about computers.
Phrases like "software brain" and "conscious machines" are ways of adding something that is not software to computers. But trust me, if we're talking about software, the programmer will know every line of their code.
Now, if you want to get meta, we can consider programs that write programs. But again, the first program will be written by a human, and we will know exactly what we are doing.
> resulting inferences more or less defy rigorous explanation
We have written programs that take input and produce data models. There is no consciousness here.
Humans take input and produce models too. But I fear you may be confusing learning with consciousness and agency. These not to be found in the code of learning machines, nor in the models they've learned. I am sorry, but it's just not there.
For every abstraction there is an implementation. That is physics. With no implementation, we have metaphysics. Here, "consciousness" is the abstraction. The physics of it is the physical implementation, and is what Roger Penrose and other scientists are working on identifying. The simulation will have a coded implementation. Meaning, whether we generate it or we build something that generates it, when we have "consciousness" we will have the code that backs it. To be able to do this without any understanding of what consciousness is would be analogous to monkeys typing Shakespeare.
Of course there is. The human "wetware" brain does just that.
>know every line of code
But code is there to simply provide some basic mechanisms, not the brain's structure. It is better compared to laws of chemistry or something on that level. It does not need to rewrite itself for the data structures that represent the simulated brain to evolve.
That said, I think you should be a little more critical of deep learning. So-called neural networks are more meaningfully described as high-dimensional curve fits. When people "train" neural networks, they're fitting the parameters of a model (just like y = a*x + b is a model with a and b as parameters) to some data. The results are impressive, and sometimes downright spooky, but neural networks are closer in spirit to a curve fit in MS Excel than they are to a real brain. There is no deep why. These are just nested compositions of affine transformations and ramp functions.
A reductionist would say that we can explain any phenomenon in the Universe because we know how atoms interact and everything is built out of atoms. When you ask why a tree grows, the answer that it grows because of the interactions between atoms is not useful and approaches the problem on too low level. Similarly a programmer can create artificial neurons and perfectly understand how these neurons operate on a low level, but it doesn't mean the programmer will understand the high level emergent phenomenons that can result from these low level interactions.
This is not at all how modern AI systems work. Modern AI systems work by feeding lots of data into a system that uses randomized heuristics to come up with an answer that has high confidence. You change the training data a little bit and the output can be entirely different, for reasons that are far from clear.
I definitely believe things happen in large programs that no one really intended.
Except that there is none: consciousness is "programmed" into our brains (by hard-wiring + social experiences); artificial intelligence is "physically reproduced" as a mesh of transistors and higher-level structures built from them - logic cells, LUTs, FPGAs, PSMs, CPUs, HPC clusters, etc.).
(Partially relying on software is not an issue: even a brain completely simulated in software could, in principle, exhibit consciousness just as well as a "wetware" one can.)
When I make a decision, I tap into my data and my knowledge, but the "I" is not inside that data nor did I emerge from it.
The topic of artificial consciousness would be Artificial Consciousness or Artificial Agency.
The ability to get things right is not to be confused with knowing what is right. Or put another way, you could be a complete idiot and still exhibit your will and perfect agency.
Seems morally relevant re: does the suffering of AI entities matter.
[1] sentience defined as feeling or sensation distinguished from perception and thought, or possibility of suffering (mental state sentient subject wants to avoid).
It's possible to conceive high level intelligence without consciousness. It's possible to conceive consciousness without sentience. Just having awareness of internal state, preferences and goals does not imply suffering.
This feels like an inadequate definition of suffering, but it's the only objective definition I can come up with. Is the conclusion that suffering is only meaningful with regards to animals? (Do insects suffer? Plants? Bacteria?).
In fact, the whole brain doesn't have pain receptors, which is why you can do surgery on it while the patient is awake.
The feelings one might experience upon hearing of the death of a close friend are clearly not the result of a "nerve transmitting pain", but most people think they're pretty unpleasant nonetheless.
If an AI can experience similar anguish, then that's ethically relevant to how we use them.
What's not entirely clear to me is how we can create useful consciousnesses without emotions in general. As humans, we are driven by our emotional states to perform different behaviors as well as think about different things.
Maybe it's possible that we can hard-code some emotional drivers into our artificial intelligences. Perhaps we could program a consciousness to be driven by the opportunity for human advancement, and allow that consciousness to take over from their in how it wishes to pursue that goal.
That's possible, but I don't see any reason to believe it's definitely true. We know essentially nothing about the nature of consciousness.
Humans can relate to humans emotionally because we are very similar to each other in a neurological way, not because we have some magic mental capability like 'sentience' or 'consciousness'.
Your syntax is suggesting that you consider yourself to be a prominent AI scientist like Andrew Ng, but I'll assume that's just a mistake!
As to what is really the aim of AI as a field, there are two issues here. On the one hand, the original AI project as conceived back in the early '50s by the likes of Turing indeed aimed at getting computers to mimic human intelligence. Everything that has happened since then in the field has followed from that original and grandiose quest.
It's true that in more recent years the applications of AI have tended to be aimed at much humbler goals, like performing specific, isolated tasks at a level approaching (and sometimes surpassing) the human- select tasks like image classification or machine translation etc.
However, this is by necessity and not by choice. Think of it this way: if we would benefit from a system that can perform select tasks as well as humans, then we would benefit a lot more by a system able to perform _all_ conceivable tasks that well (or better). In that sense, the motivation of the original AI project remains alive and well.
Consciousness of course seems to be an integral part of human intelligence- and it's reasonable to expect that it would be a prerequisite for strong AI also. In that sense, yeah, I'm afraid the discussion about consciousness is part of AI and will remain that for a long time to come.
The same can be said of any conscious experience that follows the spectrum from fear to love, terror to joy. If we want AI to solve human problems, it will need data about what it feels like to experience reality, and that data can only come by being conscious itself.
- Dijkstra
No. If it isn't conscious it isn't intelligent. There is no AI without consciousness because a non-conscious entity cannot make decisions independently -- react intelligently to its environment.
That being said, I'm not sure why there's quite so much vitriol towards Penrose and his hypothesis, the leveraging of quantum effects in photosynthesis and enzymes have been demonstrated and recent studies show that the sense of smell may also be based upon quantum phenomena. So it's not all too unreasonable that there might be something quantum going on, even if it's not Penrose and Hameroff's microtubules.
Another interesting quantum hypothesis in neuroscience is Fisher's: https://www.quantamagazine.org/20161102-quantum-neuroscience...