News for hackers. Stuff that matters.
Re:Why “Gradient” Descent? by JunaidB on Saturday January 04, 2020 @03:02AM (#21953590)
Your point about convergence to the exact minima over training loss not guaranteeing the best generalization loss reminds me of the point made in this lecture here https://www.youtube.com/watch?v=k3AiUhwHQ28.

You also made an interesting comment about work not catching on outside of the optimization community - can you recommend some resources or websites to follow in order to see what the optimization community is working on? I've developed an interest in the area but don't really know where to go for "up to date" information.

Re:Why “Gradient” Descent? by smallcharleston on Saturday January 04, 2020 @07:25AM (#21954538)
I’m not that other guy and I also haven’t read this paper but it seems quite thorough

https://arxiv.org/abs/1606.04838

Re:Why “Gradient” Descent? by JunaidB on Saturday January 04, 2020 @08:35AM (#21954816)
This seems like an excellent review. I'll check it out. Thanks very much!