We know the brain doesn't do "backpropogation" in the sense that backpropagation is a complete description of how the brain works. It may be a component, but there has to be much more, because what backpropogation can do is fairly well characterized mathematically now and is not sufficient to build anything like a human brain. If you haven't seen a paper that clearly spells that out, it's because it would be considered too trivially obvious a result to publish.

When we don't know how a thing works that does not preclude us being able to eliminate some of the possibilities. Our ignorance is profound, but not total.

This is a reply to multiple sibling comments. There is actually recent work which shows that deep learning methods can also work WITHOUT any reverse signals: http://s.yosinski.com/dan_cownden_presentation.pdf

Interesting paper. Any more details on the architecture of the feedback connections. Also I can't tell from the paper where and how weights are being updated e..g what does "train" mean in this context

I believe instead of multiplying the delta by W^t to backpropagate the error from layer l to l-1, you multiply it by a random projection B. It's hard to dig deeper because there doesn't appear to be any other information on it except here: http://isis-innovation.com/licence-details/accelerating-mach...

As an aside, are they really trying to patent a slight twist on backpropagation? That seems pretty counter-productive to me.

It looks like there are still reverse signals (eg., deltas), but they are multiplied by a random matrix B instead of the usual transpose of the weight matrix for that layer. Am I misunderstanding?

The brain could, and likely does, individual neurons do not. Synaptic current only travels in one direction.

This is not entirely correct. There is a lot of evidence that action potentials backpropagate into the dendrites [1]. On a local level, this allows synapses on the dendrites to "be aware" of the activity of the post-synaptic neuron. However, I do not think this accomplishes/implements the ANN backpropagation algorithm (although a few statements and a citation in [1] alludes to this being possible).

[1] http://en.wikipedia.org/wiki/Neural_backpropagation

Yes, that is a very interesting complication to the standard model.

As a weak epiphenominalist, I'd argue that all conscious thought is a backpropagation. Is it controversial that the brain is able to perceive its own output, or am I misunderstanding how backpropagation in neural networks is implemented?

that's not what backpropagation is. Backpropagation is best thought of as a 'cheat' (algorithmic simplification) that allows you to calculate the derivative of a feed-forward neural net. You would need to calculate the derivative of a neural net in general to optimize relative to some cost function, you could use, for example, gradient descent, but that is computationally costly.

For some neural nets, you still have a gradient, but the concept of back or forward propagation is not defineable. Based on the topology and structure of biological neural nets, what would you think is the case?

I agree. Neuroscience isn't my specialty, but I don't remember hearing about research showing that the brain does not use feedback...

Overall though, I think there is a lot of truth to his message.

Think about it; biological neurons most definitely don't use backprop. Physical neurons can't backprop linearly if they forward prop nonlinearly. I think Hinton pointed this out, though almost all ANN researchers agree.

Hebbian learning .... maybe. I never thought too much about contrastive divergence.

It doesn't really matter; brains most certainly don't work like ANNs, except maybe in some weird mean field sense for a few things like liquid state machines. It would be a huge coincidence if LeCun or Hinton or whoever magically wrote down the brain equation....