A hard problem, but back in 2008, Microsoft Photosynth could stitch together a user’s collection of images into a 3-D model. I remember seeing a detailed model of the exterior of the Notre Dame cathedral.

For people who have physical access to a property, I wonder if you could use a variant of SLAM like the robot vacs - maybe hack one of those laser tape measures to plug into your phone as you walk around the house.

Bringing a laser tape measure and a pen to a physical viewing, and just doing this manually, gets you 98% of the way.

It's not automatic of course, but you can do it pretty discretely and cover a whole house with measurements in ~ 10 minutes. I did it every single time when we were looking at houses.

Also always brought a powerful flashlight and a small spirit level.

Photosynth was a great app, really disappointed when it was discontinued. Does anyone have the backstory on that? It seemed to have a leg up on Google’s street view/cardboard app at the time.

Well if you take video or many photos of a house you can create a 3D model using photogrammetry. I was recently served an ad showing that I think Redfin includes 3D photogrammetry captures of some homes. So it’s perhaps already being done in the wild.

There are a number of companies operating in the space. Matterport is one some people in my work have been exploring for documenting projects in various stages of construction. There are definitely still a bunch of rough edges, but the results so far even just using a phone to capture the spaces have been acceptable to good.

Nice! Glad to hear it’s not just for research nerds these days. I’ve done a fair bit of experimentation with photogrammetry and it does work well for the built environment.

I have an application however where I want to be able to capture an outdoor forest scene with a video camera and convert that to a video game/simulation scene which renders efficiently. It turns out that is a more challenging problem that actually no one has solved as far as I know.

There’s lots of neat research tho and it will be possible some day. This bit of research is pretty cool: https://youtu.be/JuH79E8rdKc

This is actually a very interesting video. Reflection effects look very realistic.

Yah it’s cool stuff! Currently NERF takes a very long time to calculate and render but folks are working on improving that.

Do you think its possible to decouple lighting effects from the generated 3D scene? Currently it seem to simply copy the actual lighting condition in the environment. In a way, it's a limitation. It's much better if it can provide the RGB color + other material information (SVBRDF decomposition). If that's possible, then these meshes can be used in other environments.

I think its definitely possible, though I don’t know if the algorithms exist now. You would probably need to capture the scene under different lighting conditions, and probably from different angles. But assuming there is some theoretical way to go from photos to a mathematical representation of material characteristics I’d think we’ll eventually have neural nets that can do the conversion.

Estimating the global camera pose using non-overlapping photos is a challenging problem. This is doable if the photos densely cover the house (e.g. video frames).

However, if the global camera poses are available, we can detect surfaces in photos and project them reasonably well into 3D. In the past, I have used this work to achieve something similar: https://github.com/NVlabs/planercnn

In addition to Matterport, mentioned in another comment, there's FloPlan, which has the agent walk around the house taking a video of the baseboards, and constructs a floorplan from that. Not entirely accurate, but good enough for a rough idea of the layout of the space.

I believe it can also use the LiDAR sensor in newer phones, but I haven't tested that.

An easier sell than Matterport, which (IIRC) is a $5K camera. FloPlan just charges a per-plan price (again, IIRC).

https://floplan.io/

Matterport will also work with a phone, so the fancy camera is optional.

Floplan is based on the technology from Cubi Casa at https://www.cubi.casa/ but FloPlan is only available for MLS property listings.

If you want the generic service, you have to go to Cubi Casa.

I don't know if this is available where you live, but where I live the county assessor maintains a web application that contains information about each parcel and the buildings theron, including simple floorplans.

In Austin, Texas, you can get a “platte map” which will show the boundaries of the property and a simple outline of where the known structures fit in that map, but that doesn’t help with indoor floor plans.

A friend invented a device to do that. It was a small backpack you (or whoever is making the listing) would wear and walk into each room of the building. After gathering the data, software would turn it into a floorplan. It probably took less time per room than getting good listing photos. Not sure why she never went to market with it but I'm going to guess something similar could be done now with just a modern smartphone and the right software.

Matterport does this and uses it for real estate listings all the time.

https://matterport.com/gallery/bclm-tollhouse

The "dollhouse" view is really handy for being able to get a sense of the space.

There is a plenty of solutions like this already.

The earliest I am aware of was a specialized mini "cart" studded with video cameras that one would roll around the place and will get a textured 3D model that can then be walked around, Doom-style. It was made by a Vancouver startup was back in mid '00s.

There is also a specialized handheld "scanner" gadget that was marketed towards police use for capturing crime scene details. You'd just wave it around and see the resulting 3D phtogrammetry capture in real-time. Missed a bit - go back a wave a bit more. This was in the early '10s.

I'd imagine that this has been ported to smartphones already, because they now have sufficient sensor, storage and processing capacity for that.

> There is a plenty of solutions like this already.

I definitely feel like no one has won the market yet. Having something available or somewhat good isn't market dominance.

I wonder how it would compare to the floor plan my vacuum robot creates every time it is working. Or whether my Neato's floor plans are anywhere near realistic.

Or what Neato does with all these floor plans on their servers ...

You could try RealityCapture, https://www.capturingreality.com/, which is a professional 3d capturing software. It has been free to try since Epic bought it. Unfortunately, you'll need a lot of photos to get reliable 3d models. But playing with it will probably give you an understanding of how hard that problem really is.

So far as I can tell, the RealityCapture software only runs on Windows. You would have to feed it pictures or data from somewhere else (like your phone), but the software itself doesn’t run on the phone.

It also seems to focus on creating 3D models of things and not spaces.

The floor plans for every house should already exist in some database. Planners/builders at some point must had acquired them from the architect.

It would be nice if there was standardization for such things. No doubt most counties will have on file somewhere blueprints and drawings that were submitted for building permits but most likely haven't been digitalized. I'd hope that newer homes could have their blueprints submitted digitally by the architects but I suspect most places are still using paper centric processes.

About twenty years ago I worked for a big box store's corporate headquarters. One of my tasks was to add new stores to MapBlast so they would show up on the "Find a store" section of our website. Problem was that often stores were built in new sections of rapidly growing sprawl so the streets weren't in MapBlast yet. I'd have to make educated guesses as where to put the store on the map. Sometimes I could call the store and get some help that way but for our international stores, that was difficult because of the language barrier. We knew that there was some department somewhere in the company that had detailed drawings of each store, including precise location coordinates but the company was so large, we had no clue where to even begin to find that department.

There are some prior work that can digitize (vectorize) floorplans: https://github.com/art-programmer/FloorplanTransformation.

This can detect rooms as polygons, doors/windows as line segments and bounding boxes around objects (e.g. cabinets, sinks etc. indicated on floorplans).

Interesting startup idea to digitize the blueprints/floor plans by working with counties. I'd imagine if you offer them some money they won't refuse.