Re:Generative Adversarial Networks – The Story So Far
by gwern on Saturday June 22, 2019 @10:38PM
(#20252948)
Self-attention seems to help a lot with that. (My usual example is that with anime faces, GANs which don't use either self-attention or progressive growing frequently have a failure mode of mismatched eye color: red+blue, for example.) Self-attention is expensive, so even BigGAN uses them lightly, typically only once, like at the 64px level, but there's work on making self-attention a lot cheaper and approaches like the Sparse Transformer (which OA recently used for MuseNet to let the Transformer scale to 30k-long sequences) are promising for making self-attention a lot cheaper.
[ Parent
| on HN ]
There are no comments on this comment yet.