News for hackers. Stuff that matters.

A comment on Google A.I. researchers develop alternative architecture for neural networks.

Re:Google A.I. researchers develop alternative architecture ... by inverse_pi on Friday November 10, 2017 @11:21AM (#15670905)
The idea might be cute but performance is not there yet. Specifically they were able to achieve state of the art performance on MNIST, but got 10.6% test error on CIFAR 10 which is comparable to state of the art of 4 years ago (and if you're in the field, 4 years is like a century ago). It's important to stress that there's ABSOLUTELY NO theory backing anything so everything we're doing including this idea of capsules and dynamic routing is just brute-forcing, trial-and-error. Even though the idea is cute, there's still ALOT to be proven for this method. So when I see all these articles, I feel a little bit uneasy.
[ Parent | on HN ]
Re:Google A.I. researchers develop alternative architecture ... by zardo on Friday November 10, 2017 @01:39PM (#15671910)
>just brute-forcing, trial-and-error.

I think as a whole, the community is executing a distributed epsilon-greedy montecarlo search, which is theoretically guaranteed to converge on an optimal policy eventually.

[ Parent | on HN ]
Re:Google A.I. researchers develop alternative architecture ... by inverse_pi on Friday November 10, 2017 @05:05PM (#15673445)
not necessarily. The convergence point can be and will probably be sub-optimal if we keep doing this without a mathematical framework guiding us.
[ Parent | on HN ]
Re:Google A.I. researchers develop alternative architecture ... by zardo on Friday November 10, 2017 @05:13PM (#15673505)
So long as you never stop trying random things, you are guaranteed to find a global optimum... as time->inf

Not a very useful guarantee, but that's theoretical guarantees for you.

[ Parent | on HN ]
Re:Google A.I. researchers develop alternative architecture ... by goptimize on Friday November 10, 2017 @02:26PM (#15672263)
when time goes to infinity which is a shitty guarantee
[ Parent | on HN ]
Re:Google A.I. researchers develop alternative architecture ... by topynate on Friday November 10, 2017 @12:01PM (#15671205)
Okay, but it sounds like this technique is targeting learning from a limited number of examples. How does it compare on CIFAR 10 when restricted to ten training images per class? That's the sort of thing that you don't even see in benchmarks, because there hasn't been any way to get a handle on the problem.
[ Parent | on HN ]
Re:Google A.I. researchers develop alternative architecture ... by inverse_pi on Friday November 10, 2017 @12:36PM (#15671464)
That is not true. What you're describing is called zero to a few shot learning in the literature and there are specific benchmarks to test this. The author(s) specifically chose to NOT test on these benchmarks and there is no explicit mentions in the paper that they wanted to target a few shot learning.
[ Parent | on HN ]
Re:Google A.I. researchers develop alternative architecture ... by modzu on Friday November 10, 2017 @02:07PM (#15672120)
actually the theory comes from the brain https://en.wikipedia.org/wiki/Cortical_minicolumn
[ Parent | on HN ]
Re:Google A.I. researchers develop alternative architecture ... by inverse_pi on Friday November 10, 2017 @05:03PM (#15673434)
that's just an aspiration, not mathematical foundation.
[ Parent | on HN ]