Re:DreamFusion: Text-to-3D using 2D Diffusion
by jacobr1 on Thursday September 29, 2022 @05:59PM
(#33027864)
They reapply noise to the potentially distorted image and then predict the de-noised version like the originally rendered first frame. So the image is at least internally consistent for the frame (to the extend the the system generates consistency whatsoever).
The example with a squirrel wearing a hoodie demonstrates an interesting edge case, the "front" of the squirrel (with hoodie over the head) show a normal hooded face as expected, but when you rotate to the "back" you get another face where the hoodie is low over the eyes. Each looks fine in isolation, but in aggregate it seems like we have a two-faced squirrel.
The example with a squirrel wearing a hoodie demonstrates an interesting edge case, the "front" of the squirrel (with hoodie over the head) show a normal hooded face as expected, but when you rotate to the "back" you get another face where the hoodie is low over the eyes. Each looks fine in isolation, but in aggregate it seems like we have a two-faced squirrel.