News for hackers. Stuff that matters.
Re:Mathematics of Deep Learning [pdf] by gwern on Saturday December 16, 2017 @02:14PM (#15941277)
> Since when did we become convinced deep learning achieves "global optimality" as they put it.

There are a couple of papers over the past few years arguing/proving under various models that the local optima in deep models either are or are within epsilon of the global optima and this is why SGD reaching only local optima results in models that still work well. I assume these results are covered in OP (haven't read it yet, just answering your question).

Re:Mathematics of Deep Learning [pdf] by catnaroek on Saturday December 16, 2017 @02:32PM (#15941409)
> There are a couple of papers over the past few years arguing/proving under various models that the local optima in deep models either are or are within epsilon of the global optima

I don't care so much for the “arguing” part, but the “proving” part got me interested. ML methods reliably being able to get epsilon-close to global optima is a BFD. What mathematical technology was used to establish these results? Could you provide links to those papers?

Re:Mathematics of Deep Learning [pdf] by vardhanw on Saturday December 16, 2017 @08:44PM (#15943461)
Re:Mathematics of Deep Learning [pdf] by nabla9 on Saturday December 16, 2017 @02:40PM (#15941472)