News for hackers. Stuff that matters.
Re:'Attention is all you need' coauthor says he's 'absolutel... by mxkopy on Friday October 24, 2025 @11:03PM (#45701072)
I agree, gradient descent implicitly assumes things have a meaningful gradient, which they don’t always. And even if we say anything can be approximated by a continuous function, we’re learning we don’t like approximations in our AI. Some discrete alternative of SGD would be nice.

There are no comments on this comment yet.