The problem with noise seems to be repeatability. Enough noise, and reliability becomes a huge problem. With same inputs and analog circuit, you want the same answer every time, don't you?
You cannot "overcome this noise" by removing it. The perfection of materials, process and environment required for that would be on the order of an experiment like this: https://www.simonsfoundation.org/quanta/20131010-neutrino-ex...
We overcame the noise via digitization. In fact, there is obviously still noise in our current digital computers, since the components within them are fundamentally analog, but digital circuits quantize the analog signals, interpreting the 1's and 0's, despite their analog nature. (this is simplified and I know next to nothing about digital circuit design)
That kind of system is a huge simplification, but similar things could be done with analogue computing. In particular, I think probabilistic computing could be done by setting up network feedback loops corresponding to underlying Bayesian networks, where stable points correspond to highest likelihood parameterisations. (I may actually do some work in this direction next year, because it's pretty cool stuff.)
Computers always do exactly what you tell them, but that correctness is resource intensive. With the "approximate computing" you know you don't need an exact result because maybe you're showing it to a human who doesn't care about the difference between 3.40000 and 3.40001. What if you get that inexact result for half the power? (That's the level of claims I've seen).
That is similar to "analog computation" you might perform in your brain, e.g. recognizing a face is not 100% accurate.
Here's an article about it: http://news.rice.edu/2012/05/17/computing-experts-unveil-sup...
A bunch of more serious resources: http://shakithweblog.blogspot.dk/2012/11/approximate-computi...
"But all of the above will not get electronics down to the energy-efficiency of the brain.
"That will require many more changes, including a move to analogue computation instead of digital.
"It will also involve breakthroughs in new non-Turing models of computation, for example based on an understanding of how the brain processes information."
What exactly does all this mean? How would moving from digital to analogue computation improve energy efficiency? And aren't the whole Turing vs. non-Turing models of the brain still up for serious debate?