Worldsheet: Wrapping the World in a 3D Sheet, View Synthesis from a Single Image

Re:Worldsheet: Wrapping the World in a 3D Sheet, View Synthe... by taneq on Monday January 04, 2021 @12:32AM (#25628182)

Does anyone know what the Facebook "3D Photos" feature is doing and how/if it's different to this? Because that is generally terrible (much worse than the 'failure mode' examples here) and the depth map generated seems largely arbitrary.

[ Parent | on HN ]

Re:Worldsheet: Wrapping the World in a 3D Sheet, View Synthe... by andybak on Monday January 04, 2021 @03:33AM (#25628912)

It's worth making a distinction between "3D Photos where the user supplies depth data from the dual cameras/LIDAR etc" vs "3D Photos where Facebook's magic algorithms try to infer depth". It's the latter that I presume you're criticising here. (Might be obvious to you but I thought it worth spelling out)

[ Parent | on HN ]

Re:Worldsheet: Wrapping the World in a 3D Sheet, View Synthe... by ben_w on Monday January 04, 2021 @04:32AM (#25629203)

I think this paper is also about the latter, it’s just a much better version of the latter - the abstract says RGB image not RGBZ, and one of the examples is a painting.

[ Parent | on HN ]

Re:Worldsheet: Wrapping the World in a 3D Sheet, View Synthe... by taneq on Monday January 04, 2021 @07:20AM (#25630398)

True, and the latter is indeed what I was referring to. This looks like it does the same thing as Facebook's (train a neural net to infer depth from image contents) but it actually works fairly well.

Generating new views from an rgb+depth image is relatively straightforward and I'd expect any reasonable implementation to work pretty well.

[ Parent | on HN ]

Re:Worldsheet: Wrapping the World in a 3D Sheet, View Synthe... by moetsi on Monday January 04, 2021 @03:21PM (#25637317)

the latter was 6d.ai's whole thing. I say "was" because they were bought by Niantic

[ Parent | on HN ]