Re:Stable Diffusion: Is Video Coming Soon?
by quitit on Friday September 02, 2022 @01:52AM
(#32687069)
Piggy-backing off my comment, because I forgot to mention this very interesting project:
https://replicate.com/andreasjansson/stable-diffusion-animat...
The idea is that it takes two txt2img prompts and animates between them. The result isn't 100% there yet, but I think this idea has some legs on the 2D side of things because of the way it's able to keep a consistent composition. (One of the core issues with txt2img is that while it can produce nice images, it can product just one - animation, or even a series of story boards requires a lot of manual intervention from the creative.
Below is a mini tut where the content from Dall-e it piped to Ebsynth and then DAIN. https://www.instagram.com/reel/Ch7aV2mjWOD/
In summary: Dall-e generated the outfits, Ebsynth mapped those outfits to a range of frames (instead of having a new random artwork on each frame) & Dain smoothed the transitions between each outfit change.
Another example of this concept is here: https://www.instagram.com/reel/ChmyFNoDHZY/