News for hackers. Stuff that matters.
Re:Alice's adventures in a differentiable wonderland by xanderlewis on Tuesday April 30, 2024 @08:46PM (#40218281)
I don’t think the author literally meant tweaking the parameters and seeing what happens; it’s probably an analogy meant to give a sense of how the gradient indicates what direction and to what degree the parameters should be tweaked. Basically, substitute ‘the gradient is positive’ for ‘increasing this parameter decreases performance’ and vice versa and it becomes correct.
Re:Alice's adventures in a differentiable wonderland by p1esk on Tuesday April 30, 2024 @09:01PM (#40218390)
That substitution is the main difference between SGD and RWP.

It’s like describing bubble sort when you meant to describe quick sort. Would not fly on an ML 101 exam, or in an ML job interview.

Re:Alice's adventures in a differentiable wonderland by xanderlewis on Tuesday April 30, 2024 @10:04PM (#40218743)
It’s not like that at all. You couldn’t accidentally sound like you’re describing quick sort when describing bubble sort, or vice versa. I can’t think of any substitution of a few words that would do that.

The meaning of the gradient is perfectly adequately described by the author. They weren’t describing an algorithm for computing it.

Re:Alice's adventures in a differentiable wonderland by a_random_canuck on Tuesday April 30, 2024 @11:59PM (#40219369)
I don’t think anyone is trying to pass an exam here, but just give an understandable overview to a general audience.