Bringing a laser tape measure and a pen to a physical viewing, and just doing this manually, gets you 98% of the way.

It's not automatic of course, but you can do it pretty discretely and cover a whole house with measurements in ~ 10 minutes. I did it every single time when we were looking at houses.

Also always brought a powerful flashlight and a small spirit level.

Photosynth was a great app, really disappointed when it was discontinued. Does anyone have the backstory on that? It seemed to have a leg up on Google’s street view/cardboard app at the time.

Well if you take video or many photos of a house you can create a 3D model using photogrammetry. I was recently served an ad showing that I think Redfin includes 3D photogrammetry captures of some homes. So it’s perhaps already being done in the wild.

There are a number of companies operating in the space. Matterport is one some people in my work have been exploring for documenting projects in various stages of construction. There are definitely still a bunch of rough edges, but the results so far even just using a phone to capture the spaces have been acceptable to good.

Nice! Glad to hear it’s not just for research nerds these days. I’ve done a fair bit of experimentation with photogrammetry and it does work well for the built environment.

I have an application however where I want to be able to capture an outdoor forest scene with a video camera and convert that to a video game/simulation scene which renders efficiently. It turns out that is a more challenging problem that actually no one has solved as far as I know.

There’s lots of neat research tho and it will be possible some day. This bit of research is pretty cool: https://youtu.be/JuH79E8rdKc

This is actually a very interesting video. Reflection effects look very realistic.

Yah it’s cool stuff! Currently NERF takes a very long time to calculate and render but folks are working on improving that.

Do you think its possible to decouple lighting effects from the generated 3D scene? Currently it seem to simply copy the actual lighting condition in the environment. In a way, it's a limitation. It's much better if it can provide the RGB color + other material information (SVBRDF decomposition). If that's possible, then these meshes can be used in other environments.

I think its definitely possible, though I don’t know if the algorithms exist now. You would probably need to capture the scene under different lighting conditions, and probably from different angles. But assuming there is some theoretical way to go from photos to a mathematical representation of material characteristics I’d think we’ll eventually have neural nets that can do the conversion.