Re:Stable Video Diffusion
by mkaic on Tuesday November 21, 2023 @04:11PM
(#38370199)
I don't think transformers or diffusion models are inherently "unsupervised", especially not the way they're used in Stable Diffusion and related models (which are very much trained in a supervised fashion). I agree with the rest of your points though.
[ Parent
| on HN ]
Generative methods have usually been considered unsupervised.
You're right that conditional generation start to blur the lines though.
"Generative AI" is a misnomer; it's not the same kind of "generative" as the G in GAN.
While you're right about GANs, diffusion models as transformers as transformers are most commonly trained with supervised learning.
I disagree. Diffusion models are trained to generate the probability distribution of their training dataset, like other generative models (GAN, VAE, etc). The fact that the architecture is a Transformer (or a CNN with attention like in Stable Diffusion) is orthogonal to the generative vs discriminative divide.
Unsupervised is a confusing term as there is always an underlying loss being optimized and working as a supervision signal, even for good old kmeans. But generative models are generally considered to be part of unsupervised methods.