Humanly "imperceptible" is a very loaded term. Human perception has billions upon billions of networks worth of filtering going before we even boil down our environment to the "interesting" stuff.
Furthermore, if you take a snapshot of that network after training, you're fit to the training data. The network has lost it's plasticity. Take a potato, put it on the ground, train the network on other potato shots. Now show it a potato shaped asteroid. Now show it a French fry. What is the potato-ness that this potato+detector is ACTUALLY homing in on? Keep in mind, this structure is trained on digital encodings of maps of light and color. The function may not be a perfect semantic detector of potato-ness. It just knows what patterns of bits MIGHT be potatoes. And when you are working on bit level encodings, one bit translates to a lot of change, even if it is imperceptible to a human looking at a rendering on a screen.
Heck, there is no guarantee that the function it's emulating is well defined outside the training data set.
Neural networks are GOING to be fickle. You're trying to coerce "reliable, repeatable. generifiable results" out of a simulation of the same stuff that drives five year olds and emotional people. Consider yourself lucky the program hasn't opened the CD tray and demanded you insert crayons.
The "neural network" machine learning technique is not a simulation of biology. It's a nice marketing phrase. The technique is just math, maybe "inspired" by someone thinking about neurons.
Don't be misled by branding. For a good explanation of why some of these fanciful science terms come about, read Bellman's explanation of why he called his research "dynamic programming".
The main difference of course being that you are implementing it in silicon rather than carbon, and having no bloody grammar or conception of how in the heck to explain how 'X set of nodes with Y set of weights with Z activation threshold function'=something useful.
They called them Neural networks for a reason. It's been quite a mystery both on the biology (in terms of gray matter) and computing (in terms of ANN) front why it works at all. But it does.
I think I've read Bellman's work before, but I'll take a look. Thanks for the pointer.
Just can't find enough hours in the day to keep on top of the state-of-the-art, and unfortunately the career hasn't budged me in the direction of anywhere I could weasel work on it into my day.
C'est la vie.
The crux is that you are programming in such a manner that you can differentiate your function w.r.t. parameters which are then tuned.
This is why the 'neurones' used in ANN's are nothing like biological neurones really - both in the properties of the activation functions and the structure of the net as a whole (which is difficult as we still know relatively few details about the structure of the human brain on a neural circuitry level).