Re:DreamFusion: Text-to-3D using 2D Diffusion
by monocasa on Thursday September 29, 2022 @10:17PM
(#33030092)
> Sussman attains enlightenment
> In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6.
> “What are you doing?”, asked Minsky.
> “I am training a randomly wired neural net to play Tic-Tac-Toe” Sussman replied.
> “Why is the net wired randomly?”, asked Minsky.
> “I do not want it to have any preconceptions of how to play”, Sussman said.
> Minsky then shut his eyes.
> “Why do you close your eyes?”, Sussman asked his teacher.
> “So that the room will be empty.”
> At that moment, Sussman was enlightened.
Re:DreamFusion: Text-to-3D using 2D Diffusion
by bmpoole on Thursday September 29, 2022 @09:33PM
(#33029773)
directly training a NeRF on a single image is a terribly unconstrained problem that would lead to a volume that looks bad when the viewpoint changes. the gist of render + use diffusion model to refine is a great idea though and core to our method! the details of how to use the diffusion model for this refinement was the challenge, but once we figured that out it... just worked :)
Re:DreamFusion: Text-to-3D using 2D Diffusion
by etaioinshrdlu on Friday September 30, 2022 @03:19AM
(#33031945)
Very cool! Can we tweak your algorithm a little to be seeded with a real photo from a single viewpoint?
Re:DreamFusion: Text-to-3D using 2D Diffusion
by poolio on Friday September 30, 2022 @04:02AM
(#33032244)
it's a good idea :)
Re:DreamFusion: Text-to-3D using 2D Diffusion
by ToJans on Thursday September 29, 2022 @04:18PM
(#33026552)
"those who say it cannot be done should not interrupt the people doing it"
Although they start with random initialization and a text prompt. It seems to work well. I now see no reason we can't start with image initialization!