Worldsheet: Wrapping the World in a 3D Sheet, View Synthesis from a Single Image

Re:Worldsheet: Wrapping the World in a 3D Sheet, View Synthe... by mrfusion on Sunday January 03, 2021 @09:31PM (#25627395)

In the future you could train a model to predict the correct mesh for a given image. Then things get interesting!

[ Parent | on HN ]

Re:Worldsheet: Wrapping the World in a 3D Sheet, View Synthe... by moron4hire on Sunday January 03, 2021 @09:49PM (#25627482)

That's what they are doing. The rest of the process is not novel.

[ Parent | on HN ]

Re:Worldsheet: Wrapping the World in a 3D Sheet, View Synthe... by yorwba on Monday January 04, 2021 @04:15AM (#25629121)

They're not trying to get the correct mesh, they just treat everything as part of one big continuous surface.

That's good enough for sideways views, but fails if you try to look behind objects, because in their approach, there's no "behind", just a big, stretchy sheet covering everything.

A model with more knowledge about the world would be able to predict that a tree trunk is roughly cylindrical and not connected to the background.

[ Parent | on HN ]

Re:Worldsheet: Wrapping the World in a 3D Sheet, View Synthe... by jack_riminton on Monday January 04, 2021 @06:38AM (#25630061)

I think this is what they were referring to when they mentioned GANs

This paper does a good job of guessing what should go in a scene when given depths and shapes https://www.youtube.com/watch?v=u4HpryLU-VI&ab_channel=TwoMi...

A combination of the two would be wild

[ Parent | on HN ]