News for hackers. Stuff that matters.
Re:Deep reinforcement learning is a waste of time (2019) by unixpickle on Saturday July 10, 2021 @05:23PM (#27796565)
I think you are conflating "Transformers" and "autoregressive models". Transformers are a general purpose architecture for transforming sequences into other sequences with self-attention. AR models / GANs are frameworks for generative modeling. The model architecture is almost entirely orthogonal to the generative framework.

You can use transformers as part of GANs [1], and you can even use them as discriminative models for images [2].

[1] https://arxiv.org/abs/2102.07074 [2] https://arxiv.org/abs/2010.11929

There are no comments on this comment yet.