NeRF in the Wild: reconstructing 3D scenes from internet photography

Re:NeRF in the Wild: reconstructing 3D scenes from internet ... by airstrike on Thursday August 06, 2020 @02:32PM (#24073994)

I had this vision that one day we'll be able to reconstruct memories from our past by taking old photos and having a ML model collate everything together to form a 3D rendering of that point in time. It seems like you have gotten most of the way there.

The next step would be to have the user grab a VR headset and immerse themselves in their favorite childhood moment. One could even add avatars for loved ones using again ML-generated audio based on recordings of their voices.

Your project made me think that it wouldn't be that interesting for me to view your memories, so perhaps the best initial step for a proof-of-concept that would allow the technology to mature would be to recreate historical moments so everyone people could relive them – and they could do so entirely virtually, from the comfort of their own couch. Side note: it feels like this technology can disrupt traditional museums with the added bonus of being pandemic-proof.

Anyway, I don't really have a question... Just wanted to compliment you on this amazing work and throw this idea out there in case others want to think about it, as I'm in an entirely different field and don't have the skills and resources to make it real, and I do strongly feel this will inevitably come to life.

[ Parent | on HN ]

Re:NeRF in the Wild: reconstructing 3D scenes from internet ... by duckworthd on Thursday August 06, 2020 @03:34PM (#24074589)

That's a really cool idea! This technology does a fantastic job at reconstructing static scenes. The moving objects -- people, cars, even flora -- are out of scope here. Why? It's really hard to build a 3D model of something you only see from one direction.

[ Parent | on HN ]

Re:NeRF in the Wild: reconstructing 3D scenes from internet ... by trilinearnz on Thursday August 06, 2020 @10:28PM (#24077413)

Anyone remember this scene from Enemy of The State (1998)? https://youtu.be/3EwZQddc3kY?t=45

Jon Voight: "Can the computer take us around to the other side?"

Jack Black: "It can HYPOTHESIZE"

[ Parent | on HN ]

Re:NeRF in the Wild: reconstructing 3D scenes from internet ... by sprafa on Saturday August 08, 2020 @12:19PM (#24092784)

I was amazed at the scene at the time, and thought it was unbelievable. But then i read they had NSA advisors and maybe the US govt might have had access to some sort of primitive photogrammetry at the time?

[ Parent | on HN ]

Re:NeRF in the Wild: reconstructing 3D scenes from internet ... by airstrike on Thursday August 06, 2020 @05:16PM (#24075550)

But if we know what a car ought to look like in 3D, can't we take the one photo we have from one direction and just fill in the blanks with that a priori 3D knowledge?

[ Parent | on HN ]

Re:NeRF in the Wild: reconstructing 3D scenes from internet ... by withjive on Thursday August 06, 2020 @06:41PM (#24076190)

Similar to how GPT-3 can be applied not only to create Text, but also fill in missing pieces of Images (ie. complete the missing half of a face).

Would the logical next step, use GPT-3 to create a 3D world? :)

[ Parent | on HN ]

Re:NeRF in the Wild: reconstructing 3D scenes from internet ... by airstrike on Thursday August 06, 2020 @11:45PM (#24077964)

GPT-3D rolls off the tongue nicely

[ Parent | on HN ]

Re:NeRF in the Wild: reconstructing 3D scenes from internet ... by jhurliman on Thursday August 06, 2020 @11:38PM (#24077895)

Getting there. This is one part of the puzzle: https://arxiv.org/pdf/2007.11965

[ Parent | on HN ]

Re:NeRF in the Wild: reconstructing 3D scenes from internet ... by Impossible on Thursday August 06, 2020 @04:04PM (#24074874)

I pitched a similar idea in an interview years ago in an interview (http://www.wearegamedevs.com/2016/01/20/scott-anderson-rende...) with the added complexity of forward simulating past events with different choices. I was asked what I would do with infinite time and money though. To this day my elderly parents tell me to quit my job and work on this idea :-D.

I do believe that past time travel to reconstructed and recorded events will be one of the stickiest use cases for VR.

[ Parent | on HN ]

Re:NeRF in the Wild: reconstructing 3D scenes from internet ... by david_at on Thursday August 06, 2020 @09:33PM (#24077134)

It's funny you say that... https://news.ycombinator.com/item?id=19529921

We're hiring too. Looking for an engineer with some game dev experience.

[ Parent | on HN ]

Re:NeRF in the Wild: reconstructing 3D scenes from internet ... by stallmanite on Friday August 07, 2020 @05:08PM (#24086482)

I wish I had a more substantive comment beyond “wow” but this is really impressive. I’ve wanted something like this for a long time.

[ Parent | on HN ]

Re:NeRF in the Wild: reconstructing 3D scenes from internet ... by airstrike on Thursday August 06, 2020 @09:24PM (#24077087)

Hah! Loved the interview, thanks for sharing.

[ Parent | on HN ]

Re:NeRF in the Wild: reconstructing 3D scenes from internet ... by rasz on Friday August 07, 2020 @08:53PM (#24087998)

"I Built a REAL-LIFE Time Machine! " by Lucas Builds The Future https://www.youtube.com/watch?v=aHyNYfFfXlg

[ Parent | on HN ]