Re:AI doppelgänger experiment – Part 1: The training
by pbhjpbhj on Monday May 20, 2024 @01:39PM
(#40417959)
AIUI diffusion models of images are trained to make a noisy image into an image without noise (reverse diffusion). Then when an image is required, the system starts with white noise, and maybe a prompt, and then iteratively performs reverse diffusion - guided by the prompt - to produce a final image.
You can enhance an image by [forward] diffusing it, then feeding it to the NN created to perform the reverse diffusion.
"Generative AI" isn't a single technology. Text and image generation don't actually work the same way (have the same model architecture), which frankly makes it absurd we invented them both at the same time.
But for text at least this isn't true; LLMs have fractal complexity, which is why they work. ("Average" text would just be the letter 'e'.) It's harder to say how diffusion models even work.
It makes them easier to handle if you train them to act like this though, but try playing with a "base model" and you'll see all kinds of inhuman results.