Sora: Creating video from text

Re:Sora: Creating video from text by Animats on Thursday February 15, 2024 @08:33PM (#39391938)

I suspect that one of the first applications will be pre-viz. Before a big-budget movie is made, a cheap version is often made first. This is called "pre-visualization". These text to video applications will be ideal for that. Someone will take each scene in the script, write a big prompt describing the scene, and follow it with the dialog, maybe with some commands for camerawork and cuts. Instant movie. Not a very good one, but something you can show to the people who green-light things.

There are lots of pre-viz reels on line. The ones for sequels are often quite good, because the CGI character models from the previous movies are available for re-use. Unreal Engine is often used.

[ Parent | on HN ]

Re:Sora: Creating video from text by becquerel on Friday February 16, 2024 @06:40AM (#39395675)

This is a fascinating idea I'd never considered before.

[ Parent | on HN ]

Re:Sora: Creating video from text by theshrike79 on Friday February 16, 2024 @04:54PM (#39403479)

Especially when you can do this with still images on a normal M-series MacBook _today_, automating it would be pretty trivial.

Just feed it a script and get a bunch of pre-vis images for every scene.

When we get something like this running on hardware with an uncensored model, there's going to be a lot of redundancies but also a ton of new art that would've never happened otherwise.

[ Parent | on HN ]