News for hackers. Stuff that matters.

A comment on Any Deep ReLU Network Is Shallow. Parent comment

Re:Any Deep ReLU Network Is Shallow by charliea0 on Friday June 23, 2023 @05:34PM (#36452445)
Yes - or Fourier series or other decomposition into a sum of orthogonal basis functions.
[ Parent | on HN ]
Re:Any Deep ReLU Network Is Shallow by mkl on Friday June 23, 2023 @09:41PM (#36454599)
I don't think orthogonality is important here. Taylor series basis functions are not orthogonal, and nor are ReLUs.
[ Parent | on HN ]