News for hackers. Stuff that matters.

A comment on Do vision transformers see like convolutional neural networks?.

Re:Do vision transformers see like convolutional neural netw... by mrfusion on Wednesday August 25, 2021 @12:24PM (#28303591)
It makes sense to me that attention would be hugely beneficial for vision tasks. We use contextual clues every day to decide what we’re looking at.
[ Parent | on HN ]
Re:Do vision transformers see like convolutional neural netw... by sillysaurusx on Wednesday August 25, 2021 @12:26PM (#28303617)
It may make sense, but it also makes no sense. CNNs already have full view of the entire input image. That's how discriminators are able to discriminate in GANs.

We added attention and observed no benefits at all in our GAN experiments.

[ Parent | on HN ]