Re:Imagen Video: high definition video generation with diffu...
by simonw on Wednesday October 05, 2022 @02:27PM
(#33099366)
"These tools just regurgitate a mishmash of images from it's training set"
I don't think that's a particularly useful mental model for how these work.
The models end up being a tiny fraction of the size of the training set - Stable Diffusion is just 4.3GB, it fits on a DVD!
So it's not a case of models pasting in bits of images they've seen - they genuinely do have a highly compressed concept of what a cactus looks like, which they can use to then render a cactus - but the thing they render is more of an average of every cactus they've seen rather than representing any single image that they were trained on.
But I agree with you on taste! This is why I'm most excited about what happens when a human with great taste gets to take control of these generative models and use them to create art that wouldn't be possible to create without them (or at least not possible to create within a short time-frame).
I don't think that's a particularly useful mental model for how these work.
The models end up being a tiny fraction of the size of the training set - Stable Diffusion is just 4.3GB, it fits on a DVD!
So it's not a case of models pasting in bits of images they've seen - they genuinely do have a highly compressed concept of what a cactus looks like, which they can use to then render a cactus - but the thing they render is more of an average of every cactus they've seen rather than representing any single image that they were trained on.
But I agree with you on taste! This is why I'm most excited about what happens when a human with great taste gets to take control of these generative models and use them to create art that wouldn't be possible to create without them (or at least not possible to create within a short time-frame).