Below is a mini tut where the content from Dall-e it piped to Ebsynth and then DAIN. https://www.instagram.com/reel/Ch7aV2mjWOD/
In summary: Dall-e generated the outfits, Ebsynth mapped those outfits to a range of frames (instead of having a new random artwork on each frame) & Dain smoothed the transitions between each outfit change.
Another example of this concept is here: https://www.instagram.com/reel/ChmyFNoDHZY/
https://replicate.com/andreasjansson/stable-diffusion-animat...
The idea is that it takes two txt2img prompts and animates between them. The result isn't 100% there yet, but I think this idea has some legs on the 2D side of things because of the way it's able to keep a consistent composition. (One of the core issues with txt2img is that while it can produce nice images, it can product just one - animation, or even a series of story boards requires a lot of manual intervention from the creative.
There have been some actual video explorations of SD's latent space, this example amazes me: https://twitter.com/karpathy/status/1559343616270557184
Presumably you'd want to be director and be able to control camera angles. You might want to have a cartoon or 3D render visual style. Unreal engine prompt tag even more literal. Most important, you will want actors. Anything alive will have to move convincingly and intentionally. Some might even bring up the ethics of allowing the movie to end.
Before proper video, I think we'll first have to see tools for music, 3D rendering and animation that bring down difficulty by orders of magnitude.
The fact that no one has been able to demonstrate it convincingly makes me think that it might actually be quite hard.
Something else which would be possible is the use of a model like SD in combination with a frame interpolation model like [2] as a video generator. Use SD to generate key frames, feed these to FILM and let it generate the intermediate frames and you should get video.
[1] https://twitter.com/SALT_VERSE
[2] https://film-net.github.io/