News for hackers. Stuff that matters.
Re:Understanding gradient descent by jofer on Friday August 05, 2016 @11:05AM (#12232851)
Gradient descent is for non-linear problems where you can't directly invert or somehow linearize the problem.
Re:Understanding gradient descent by pmarreck on Friday August 05, 2016 @11:47AM (#12233197)
Often you can directly invert, but with current methods it would just be prohibitively expensive computationally.

What I'm trying to say is that occam's razor seems to suggest that there is a better way to do matrix inversions which we simply haven't discovered yet.

Re:Understanding gradient descent by dragonwriter on Friday August 05, 2016 @11:59AM (#12233308)
> What I'm trying to say is that occam's razor seems to suggest that there is a better way to do matrix inversions which we simply haven't discovered yet.

Occam's razor doesn't suggest that. Such a method is an additional assumed entity that adds nothing to our ability to explain what is observed, and is thus the kind of thing Occam's razor calls one to reject, rather than suggests.

Now, belief in the existence of such a method might be a means of satisfying an aesthetic preference for a computationally convenient universe, but that's something very different than Occam's razor.

Re:Understanding gradient descent by pmarreck on Friday August 05, 2016 @02:13PM (#12234582)
Fair enough, you got at what I was getting at in your last paragraph