To make the same point differently, if you wanted to replicate human intelligence in its entirety (rather than creating something capable of performing some well-defined task), AI is far too high a level of abstraction. You might be able to replicate a universe and wait for intelligence to evolve, but you still wouldn't understand the intelligence you had created.
The danger of the hardware/software analogy is that we deliberately separated computer systems into (mostly) well-defined and cleanly separated layers. Given how hard it is to avoid leaky abstractions in computer systems that are: (a) vastly less complex than human intelligence and (b) designed to be layered, the systems responsible for human intelligence probably don't expose the clean API required for us to understand them using high-level concepts.
I mean, consider the solar system. Can the solar system be modeled by a computer? Yes of course. Is the solar system a computational system? Is it organized on computational principles? Of course not. These questions are not the same thing, and yet conversation about the latter always seems to get drowned out by people saying "How could you possibly deny the former!"
"human minds exist"
Therefore, whatever it is that minds do is possible.
Religions might be right and the mind is actually the soul.
Or we could all be characters in a virtual world "played" by beings operating us. This world could be an elaborate RPG.
Or something else we haven't figured out yet.
So don't confuse your belief with proof. Your argument has demonstrated nothing except a failure of imagination.
Even if minds need unknown 'laws' emerging from dualistic nature of existence, they still exist.
Whether we are human beings in a world that is as it seems, characters in an RPG game, brains in a vat, souls tortured by a demon, artificial minds somewhere in a cascade of simulations or dreaming Hindu gods, we are (or rather I am) still minds under the des cartes definition of a mind and that mind exists.
As for not confusing proof with belief… Occams razor has value. Even if the religious view of souls or such is possible, it is not based on any knowledge or information. It's made up. It has a kind of a longshot chance of being correct in the same way early philosophical claims that matter is made up of wind, fire, earth and water were had a longshot chance of being correct.
In any case, minds seem to have a lot to do with brains and brains are nice squishy physical things that can be prodded and examined and haven't yet been seen to do stuff that is impossible in a material universe.
It's quite possible that we can't engineer a computer that does the same thing as the human mind because of our own limitations - the primary one being time.
That's no reason not to try though.
*We can build one" is not necessarily the conclusion unless we're accepting pretty abstract use of the terms we and can.
To build a universe inside this universe, universes must be capable of containing universes. To build one outside it, stuff must be capable of existing outside of it. OTOH, it seems like multiple minds can and do exist. It gets pretty loopy (and awesome) if we keep going down this rabbit hole.
The statement is intentionally tautological, or very very close to it. If minds exist then minds exist so there's no point in arguing that they can't exist.
I really wish Douglas Adams or Alan Watts or one of those guys would jump in this thread and explain.
Actually, this analogy is the foundation of our modern understanding of the mind and is accepted by nearly all experimental Psychologists/Cognitive Scientists. It is certainly pervasive enough to be arguing against. Few of them seriously recognize its limitations.
This analogy is not the justification for why people believe that "the mind can be replicated on a computer". And yet the article tries to disprove the latter, deep and meaningful point by attacking the relatively superficial analogy between [current] computers and minds.
I think you're missing the point. No one cares whether the mind can be replicated on a computer. Researchers/theorists care about whether the mind is a computer. The solar system can be modeled on a computer, and yet there isn't an entire theoretical approach to astronomy predicated on uncovering the algorithms running on the solar system computer. This line of discussion about the capabilities of math and computation are beside the point.
But there is no evidence that we can do that. Maybe it is possible, but there is as yet zero data, nor a proven theory that says computation leads to consciousness. Of course, you might say that is like attributing a certain magical underpinning to consciousness and there is no reason to believe magic exists. But is there a logical, rational, tested theory as to why a certain level of computation should and will lead to consciousness?
It's a claim that is made, I think, with a simular amount of emotional and irrational vigor as those that say that AI will never be able to mimic the brain/mind/consciousness.
Personally, I think, if it is possible, it won't happen any time soon.
And again. If "dumb" matter can become aware (us) what's to hinder AI to become?
The real answer is still. "We don't know."
I think of maths as a bit of a mission to carefully extract the parts of the computational universe that can be reasoned about. There's heaps more kinds of processes that still run just fine (on a Turing machine, say) but whose outcome can't be predicted using less work than any other universal machine would take.
Bit off topic I guess but to me, this idea has great potential for a basis of the aspect of consciousness that's like magic. It could be irreducibly complex and inconsistent computation. Still perfectly valid, but un-analysable.
For example, notice that the content of your conciouness is the only thing that's certain. Law of physics can change. Or you might wake up from matrix (recursively multiple times). Basically anything can happen. But the fact that you feel good (or bad) at this moment is certain. Mind is something like a window to the physical world.
Also, consciousness is subjective, physics cannot describe it, not even in theory.
Tell that to someone with multiple personalities. Tell that to someone who has brain injuries reducing them to a vegetable. Trauma applied to the brain directly and (to an extent) predictably causes changes to the consciousness, making its contents anything but certain.
(Imagine a device that projects some unknown field at you and can predictably manipulate your subjective experience; it must have a sufficiently complete model of your consciousness and it is difficult to say much more than it is unlikely to be possible)
So far, everything in reality seems to be "made of" either determinism or randomness. There might be a third option (and options beyond that), but we haven't seen one or thought of one yet.
So far, computation can model determinism perfectly well (after all, computation is just very orderly, bottled-up determinism harnessed to a predictable human purpose!), and model randomness as well by drawing random bits from an entropy source. When you get deep enough, mathematics is in fact reducible to computation (paging Robert Harper). Given this, the Church-Turing Thesis says that our mechanical computers ought to be able to model anything reality is made of: either reality is theoretically Turing-recognizable given enough compute-power, or reality contains super-Turing phenomena that we could exploit to build super-Turing computers.
Thus, if we accept a couple of assertions that have held up decently so far, it looks as if all parts of our prosaic, everyday reality ought to be amenable to computational modeling, provided we find the right model and throw enough flops at it.
They are assertions, but they've also held up for a damn long time, and we would need some extraordinary evidence to accept the extraordinary claim that those assertions don't hold.
Note that the formulation of these are counter-examples and thus assert a fundamental limit of computation, but it says nothing that there might be many other things that are non-computable that do not fit the form of these counter-examples.
I've heard it referred to as the Church–Turing–Deutsch principle [1], since Deutsch talked about the physical version of it.
I think it's interesting that the assumption that nature is computable comes so naturally, even though as far as I can tell it's entirely unclear whether it is or not. In particular, Deutsch points out that if the universe needs reals that includes a whole bunch of noncomputable numbers, and in fact our favorite model of physics for a significant chunk of history ('classical physics') did require reals and so was noncomputable.
I also tend to assume that nature is computable, but I don't think these assertions have really "held up for a damn long time".
As well as the fact that we managed with not much of a model followed by a noncomputable one for most of history, it seems to me that there might be a bias towards finding computable models over noncomputable models, which makes any reasoning based on observing that we've found a bunch of computable models suspect.
[1] https://en.wikipedia.org/wiki/Church-Turing-Deutsch_principl...
"When you get deep enough, mathematics is in fact reducible to computation (paging Robert Harper)."
Could you elaborate on that, as it seems to be a big jump? I'm not familiar with Harper's work (I'm assuming you mean this [1] Robert Harper). I'm also not a mathematician...
[1] http://en.wikipedia.org/wiki/Robert_Harper_(computer_scienti...
[2] Or, presumably, Lambda calculus or Godelian recursion.
http://existentialtype.wordpress.com/2011/03/27/the-holy-tri...
http://www.doc.ic.ac.uk/~gds/PLMW/harper-plmw13-talk.pdf
Types are propositions! Programs are mappings! Computation is proof!
These are the truths of this world! Submit to those truths, you webdevs in human clothing ;-)!
(I'm always hoping someone will catch the obscure references I throw into my comments, but alas.)
This analogy is not the justification for why people believe that "the mind can be replicated on a computer". And yet the article tries to disprove the latter, deep and meaningful point by attacking the relatively superficial analogy between [current] computers and minds.
To me, the reasoning behind why I think a computer can theoretically do anything a brain can is much simpler. Programs are ultimately nothing more and nothing less than mathematics. Their capabilities are a finite (but theoretically arbitrary) subset of what mathematics models. And so far, mathematics has been able to model everything. Purely inductively, it seems mathematics should be able to model the mind just like everything else. After all, there is nothing magical about the brain!
At the limit of plausibility, we could just reproduce a mind by modelling a brain using the laws of physics. Sure, this is wildly impractical with current hardware. Perhaps we will never have the hardware to make it practical. But it also demonstrates how it should be possible theoretically: and if something is possible, no matter how impractically, there's a reasonable chance that it's possible in some significantly easier ways.
Saying that a computer can never fundamentally replicate a human mind is the same as saying the mind is beyond mathematical description. To me, this idea does not make sense. I'm not even sure it's well-formed! Mathematics is not a finite set of ideas or a bounded language: it is more like a standard of formality which we can extend in an incredible amount of different ways.
Sure, math has some internal limitations like incompleteness. But these are not really absolute bounds, at least not in the most obvious way. One thing to keep in mind is that mathematics is purely self-contained: all that matters is internal consistency, not a parallel to anything in our actual universe. This means the fundamental limitations are inevitably meta-limitations: they come up when we turn math on itself. Moreover, they only come up when we start encountering the infinite. The human mind may present some meta facets, but it is not infinite, so incompleteness cannot prevent us from modelling it.
It's interesting to note that all the actual arguments for why a computer cannot represent the mind are not actually logical themselves. They rely on vague analogies and emotions, quoting Ionesco but not providing actual reasoning.
Ultimately, it feels like this article is trying to establish a false balance between the two sides of the argument. But they are not anywhere near equal: the people that don't think a computer can model the mind need to present an actual case for why it's beyond the capabilities of mathematics, the system which has fared so well in describing everything else. Where does the line between physics and cognition (which exists on a purely physical plane) come from? Why are they divided? To me, that line simply seems to arbitrary to accept without more cause.
I can see why this line of thought is appealing: consciousness certainly feels magical, so you want to find some real magic there. But I am not convinced real magic exists. Or even makes sense. It would require the human mind to somehow be special in a way that would, at best, be a really suspicious coincidence. I'm not buying it.