News for hackers. Stuff that matters.
Re:Interactive Gradient Descent Demo by clix11 on Tuesday November 16, 2021 @12:36PM (#29243550)
> what advantage do I get from the compute cost of gradient descent over random sampling?

Random sampling becomes prohibitive in higher dimensions due to the curse of dimensionality [0]. Gradient descent doesn't have this problem and will always converge to a local (but, as can be seen here, not necessarily an absolute) minimum.

The step size effectively controls how far from the "real" local minimum you can get: too big a step size and you end up repeatedly "jumping over" the minimum.

[0] - https://en.wikipedia.org/wiki/Curse_of_dimensionality

Re:Interactive Gradient Descent Demo by motohagiography on Tuesday November 16, 2021 @03:37PM (#29246068)
Perfect, that's the problem I needed to know about! I had been using combinatoric explosion as an example for how to explain why some tasks were harder/impossible, but it's a subset of this more general information problem, thank you.