Neural networks, Support Vector Machines, Hidden Markov Models and other stuff (Markov Networks, etc) do something like linear regression on some huge space - they draw a curve/plane between groups of things on this feature space. The tendency is for this division to make sense and to correspond to our common sense categorization of these things.
The problem is that once that happens, you can't really reason about the division you've drawn. It's just there. You can tweak for various purposes but that's a manual process.
You categorize animals by shape or particular adaptation or by genetic makeup. You can teach one of these algorithm each of these categorizations. But you can't do something like have the thing categorize for one purpose and then tell it to "change it's outlook" and categorize for a different purpose.
In this sense, despite seeming impressive, the products of these processes are dead-ends that we can't reason about, that lack the flexible intelligence of a human being.
In other words, the incomprehensibility of a modern AI model is not a failing of AI, it is a failing of (AI) psychology and (AI) neuroscience.
The artificially constructed intelligence works whether or not we understand how. The frontier of AI science is now open to AI Psychology. Psychologists and Neuroscientists will replace the data scientists! Such fun!
The artificially constructed intelligence works whether or not we understand how
The operation of human intelligence is very dependent on the fact that we human have operational understanding of each other's mental processes. Moreover, it is well known that human beings process language and can re-evaluate past experience in light of present understand.
Oppositely, we know enough about the properties of the various non-linear recognizers to know that constructs can't do and won't ever be able to do these things.
To debug a program you actually verify that it's logic is correct (at least the good kind of debugging).
Consider a spectrum:
1. Natural language - we humans combine fragments of natural language easily and on an ad-hoc basis. We can get a fairly amount of use from reusing Shakespeare quotes and neologisms while spending rather little effort.
2. Trained programmers can reuse and combine general purpose libraries - with difficulty and often after considerable debugging.
3. AI algorithms like Neural Networks. These are just plopped in and tweaked, not combining seems possible.
It seem like "intelligent behavior" should be going more towards #1 but the process of Machine Learning seems to move things more towards #3. The "learn once, understand never" approach means that for each significant case, you'll need to do a re-tweaking and re-learning. The potential to get harder rather than easier over time might well be there.
Is a human brain intelligent? I believe so.
Admittedly, all this is in manner of speaking but still, I would claim that most if not all the times you debug a program you are also debugging your mental concept of what the program does. By that fact that we can change our concepts, our minds are very "debuggable."
An example of doing facial reconstruction: http://deeplearning4j.org/facial-reconstruction-tutorial.htm...
This is a fundamental Question: Can we really say and predict what a neural network does?
Contrary to an engineered / constructed algorithm a neural network is 'trained'.
Whenever we will present a 'known' input pattern it will reposnd with a 'learned' response.
This however introduces interesting problems: How can we _debug_ a neural network? How can we debug a correlation? Sure we can tune its paramters, we can train it some more to again show the desired response. But reaching this point we just abandoned knowing how the intrinsic algorithm works in favor to just focussing on the result.
Okay - now if we follow this argument - this would lead to: If we simulate the whole brain by simulating the neural network, we won't gain any knowlege about the intrinsic workings of the brain. We won't find any enlightenment about the innermost algorithm _represented_ by the neural network we call our brain.