Stable Diffusion animation

Re:Stable Diffusion animation by redox99 on Wednesday August 31, 2022 @04:20AM (#32659685)

For 3D we would probably need something like Blender or similar, because at some point it's just easier to use a 3D software to pinpoint where you want stuff to be, than try to use words.

Imagine opening blender and typing

> A medium sized classroom, well lit, with two blackboards and many geography posters

And the AI just generates all the 3D meshes and places them appropriately.

Repeat that for other props or characters that you need. After that you can manually tweak the scene as you currently would (moving things, etc).

Then you select a character, and to animate you tell it

> The character calmly walks to the door and proceeds to open it

You could literally do a 100+ hour job in 5 minutes.

[ Parent | on HN ]

Re:Stable Diffusion animation by danwills on Wednesday August 31, 2022 @08:14AM (#32661142)

I think you're right about the possibilities here and I love the idea and have thought similar things myself too, but to me the inclusion of a 3D element should probably be a format, not necessarily locked into any specific app such as Blender. Maybe (Pixar) USD is one possible format that could be used for general 3D interchange for this kind of thing?

[ Parent | on HN ]

Re:Stable Diffusion animation by redox99 on Wednesday August 31, 2022 @11:33PM (#32671612)

The blender thing was just as an example. Of course it would be possible to convert whatever the output of the model is, to whatever software you want to plug it in.

[ Parent | on HN ]