Re:We Found an Neuron in GPT-2
by candiodari on Friday February 17, 2023 @07:57AM
(#34834088)
> CNN are basically the simplest way to encode translational invariance
That's the geometric deep learning theory, isn't it? Do you know if there's a list somewhere of exactly what invariance has which ways to simulate it? Like an overview?
In the case of CNN the reason it works is that an image of an object X is still an image of object X if the X is shifted left or right. The property is translationally invariant. CNN are basically the simplest way to encode translational invariance.