News for hackers. Stuff that matters.
Re:Any Deep ReLU Network Is Shallow by alexfromapex on Thursday June 22, 2023 @02:34PM (#36436701)
It seems intuitive since ReLU is just a type of implicit regularization. Why would subsequent gradient descent help once you've achieved the benefit of throwing away the "outliers" or data beyond the threshold you want?

There are no comments on this comment yet.