Re:Any Deep ReLU Network Is Shallow
by alexfromapex on Thursday June 22, 2023 @02:34PM
(#36436701)
It seems intuitive since ReLU is just a type of implicit regularization. Why would subsequent gradient descent help once you've achieved the benefit of throwing away the "outliers" or data beyond the threshold you want?
[ Parent
| on HN ]
There are no comments on this comment yet.