Re:The Principles of Deep Learning Theory
by throwaway693 on Saturday April 16, 2022 @09:17AM
(#31052071)
I hear this phrase everywhere "Every layer is a change in representation of the previous layer". Is this mathematically proven or is this an assumption looking at the image classification models layer?
[ Parent
| on HN ]
Re:The Principles of Deep Learning Theory
by karpierz on Saturday April 16, 2022 @09:27AM
(#31052142)
That seems true by definition, as each layer is a functional transformation of the previous layer.
I think want they want to imply is "Each layer is a semantically meaningful representation, changed from the previous layer" which I don't think is a verifiable claim (as what is semantically meaningful isn't an objective claim)
Re:The Principles of Deep Learning Theory
by euphetar on Saturday April 16, 2022 @09:28AM
(#31052155)
Well since the next layer outputs are a linear transformation plus some nonliearity of the previous function, it's a fact that it's a change in representation. But I guess the true question is broader: "Is it proven that the next layer is preparing a feature representation for the next one?".
I don't know if it is mathematically proven, but you can easily see it yourself when making an image classifier with one hidden layer. It has some really good evidence for an assumption, at this point I would call it at least empirical evidence.