News for hackers. Stuff that matters.
Re:More Descent, Less Gradient by klipt on Wednesday April 29, 2020 @12:42PM (#23021187)
Yeah I remember being told that quasi-Newton methods can get close to the final answer in just one step in my undergraduate statistics class.

But what works for traditional statistics (like matrix inversion) tends to have the wrong big-O for big data. Would be nice though if there were a website somewhere that summarized the state of the ML-applicable optimization field with links to papers in order of increasing complexity. Instead there are like a billion websites rehashing basic SGD that seem to dominate search results.

Although I guess most of the easy to use methods like Adam are already baked into all the frameworks.

Re:More Descent, Less Gradient by benrbray on Wednesday April 29, 2020 @03:59PM (#23023480)
Just like you say, I've also recently found it much harder to google for survey articles too. I know they do exist somewhere, but I can't seem to find the magic words I need to enter into the search bar!

We seem to be returning to a world where the only reliable way to find information is to stumble upon someone's webpage (e.g. [1]) who has curated a list of links.

[1]: http://www.math.columbia.edu/~khovanov/resources/