News for hackers. Stuff that matters.
Re:A New Twist on Neural Networks by dr_zoidberg on Thursday November 02, 2017 @08:50AM (#15609898)
Hinton explains the concept of capsules in this video: https://www.youtube.com/watch?v=rTawFwUvnLE

Which is a lot better than reading someone tell you about this new idea that is called "capsules" but doesn't go into detail. The only thing is that, when this presentation was given, it seems they hadn't worked much more than MNIST (so the new thing now would be the toys-recognition net).

Better source, with date: http://techtv.mit.edu/collections/bcs/videos/30698-what-s-wr... (December 2014, for the lazy).

Re:A New Twist on Neural Networks by pmikesell on Thursday November 02, 2017 @12:07PM (#15611621)
Thank you for the link to the video - this was excellent. His key critique of convnets is that (paraphrasing) all of these pooling layer (max, avg, whatever) are really a very poor way of attempting to solve what is actually a routing problem.
Re:A New Twist on Neural Networks by olewhalehunter on Thursday November 02, 2017 @09:12AM (#15610065)
at a first glance this looks like it's going back to old 80s research on neural representations, back then this kind of stuff was known as Parallel Distributed Processing

https://mitpress.mit.edu/books/parallel-distributed-processi...

Re:A New Twist on Neural Networks by grandalf on Thursday November 02, 2017 @12:25PM (#15611834)
Indeed. A blast from the past.

Oddly the capsule approach is how I naively thought image recognition worked until I learned more about it.

Re:A New Twist on Neural Networks by SophosQ on Friday November 03, 2017 @03:40AM (#15617310)
The only thing is that, when this presentation was given, it seems they hadn't worked much more than MNIST (so the new thing now would be the toys-recognition net).

The paper however details their experiments on Cifar-10 and other datasets in the discussion section.

They don't produce good enough results but the paper proposes certain hypotheses for the poorer performance and that it could be overcome in the future.

Arxiv-link: https://arxiv.org/abs/1710.09829

Re:A New Twist on Neural Networks by thomasahle on Thursday November 02, 2017 @06:21PM (#15614834)
Apparently, one of the main changes since the talk, is that then he used a "probabilistic unit" next to his "property vector". Now the length of the property vector indicates the probability.
Re:A New Twist on Neural Networks by mannigfaltig on Thursday November 02, 2017 @06:45PM (#15615020)
In their second publication they actually use the probabilistic unit, but computing those requires running an EM algorithm for each layer of capsules: https://openreview.net/forum?id=HJWLfGWRb