News for hackers. Stuff that matters.

A comment on Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others. Parent comment

Re:Machine-Learning Maestro Michael Jordan on the Delusions ... by sherjilozair on Tuesday October 21, 2014 @01:53AM (#8486111)
This is a reply to multiple sibling comments. There is actually recent work which shows that deep learning methods can also work WITHOUT any reverse signals: http://s.yosinski.com/dan_cownden_presentation.pdf
[ Parent | on HN ]
Re:Machine-Learning Maestro Michael Jordan on the Delusions ... by jimduk on Tuesday October 21, 2014 @05:24AM (#8486587)
Interesting paper. Any more details on the architecture of the feedback connections. Also I can't tell from the paper where and how weights are being updated e..g what does "train" mean in this context
[ Parent | on HN ]
Re:Machine-Learning Maestro Michael Jordan on the Delusions ... by asdavis on Tuesday October 21, 2014 @12:23PM (#8488101)
I believe instead of multiplying the delta by W^t to backpropagate the error from layer l to l-1, you multiply it by a random projection B. It's hard to dig deeper because there doesn't appear to be any other information on it except here: http://isis-innovation.com/licence-details/accelerating-mach...

As an aside, are they really trying to patent a slight twist on backpropagation? That seems pretty counter-productive to me.

[ Parent | on HN ]
Re:Machine-Learning Maestro Michael Jordan on the Delusions ... by asdavis on Tuesday October 21, 2014 @12:25PM (#8488107)
It looks like there are still reverse signals (eg., deltas), but they are multiplied by a random matrix B instead of the usual transpose of the weight matrix for that layer. Am I misunderstanding?
[ Parent | on HN ]