News for hackers. Stuff that matters.

A comment on Any Deep ReLU Network Is Shallow.

Re:Any Deep ReLU Network Is Shallow by PaulHoule on Thursday June 22, 2023 @08:40AM (#36431246)
It doesn’t surprise me. It’s been known a long time that you can model arbitrary functions with a 3-layer network with logistic activation.
[ Parent | on HN ]
Re:Any Deep ReLU Network Is Shallow by RobbieGM on Thursday June 22, 2023 @01:36PM (#36435799)
Do you have any further information or a source for this? As someone unfamiliar with ML, this sounds crazy to me.
[ Parent | on HN ]
Re:Any Deep ReLU Network Is Shallow by hazrmard on Thursday June 22, 2023 @02:59PM (#36437059)
It was first proposed here[1]: "Approximation by superpositions of a sigmoidal function"

[1]: https://link.springer.com/content/pdf/10.1007/BF02551274.pdf

[ Parent | on HN ]