Re:Diffusion Models Beat GANs on Image Synthesis
by doctorhandshake on Wednesday May 12, 2021 @06:57AM
(#27128841)
What are the manual / human-guided techniques you speak of?
Re:Diffusion Models Beat GANs on Image Synthesis
by sillysaurusx on Thursday May 13, 2021 @09:51AM
(#27142065)
I wanted to do a thorough writeup, but I never got around to it. Here are a bunch of examples of me using those techniques though: https://twitter.com/theshawwn/status/1182208124117307392
The dota community seemed to like it. :)
https://www.reddit.com/r/DotA2/comments/dfv0z3/im_making_a_n...
Basically, it was an interactive editor where you could slightly move along stylegan directions, combined with Peter Baylies' reverse encoder to slightly morph the image to a specific face on demand.
It was instantly so much better than any automated solution. It felt like being a pilot in front of the controls of a cockpit.
The article you want is probably this one: https://ajolicoeur.wordpress.com/the-new-contender-to-gans-s...
I would launch into a thorough explanation, but it would likely not be correct in every detail, because it's been around nine months since I was immersing myself in DDPM type models. But, broadly speaking, with normal training, your goal is to train a model (show it a bunch of examples) until the model can guess the right answer most of the time in one try. Except, "guessing the right answer" is actually an easier problem than generating an image, because the model usually gives you its top-N guesses, so it says "I think it's a snake or a dog or an apple."
Whereas with generative images, it's much harder to come up with a technique that can be "sort of correct": if you generate a stylegan image, it either looks cool or looks like crap, and it's rather difficult to automatically take a crappy output and turn it into something that looks cool. (The "automatic" is key; there are manual human-guided techniques that I'm quite fond of, and amazed no one's turned it into a photoshop-type plugin yet, but the field of ML seems to compete/care about fully automatic solutions right now. For some reason.)
DDPM is the inverse: you have a trained model, and you start with noise, but then it gets progressively closer to a cool looking result by searching multiple times (i.e. taking multiple forward passes). That's as much as I remember, I'm afraid.