* Mapping the physical world

Ever wanted a floor plan for your home? Just wave your phone around.

* Visual annotation

Add direction overlays, see the plan for a play your team is executing on the helmet HUD, highlight "dangerous"(weaving, too fast, whatnot) drivers on the car HUD.

* Integrate sensor data to extend human perception

Add an IR overlay. Sample sound across the room, do a volumetric display of noise levels. "see" the strength of your WiFi signal.

* Image post processing

You have a 3d map of an area, plus pictures of all textures - rearrange to your hearts content.

* Alternate Reality

Completely change the look of the world around you, just because you can. (Semi-useful application: Interior decoration. See that couch right in your living room before you buy it)

There are tons of applications there. It mixes the "reality" of physical space with the malleability of the digital space.

I think I'm a neo-luddite.

I don't think you're a luddite. With any technology like this, it's important to ask: what applications does this really enable and will those applications simply be a waste of time.

This is an unveiling of a technology but the "applications" they show in this video are probably a waste of time to most users. Google have not shown a killer app that uses this tech. Maybe they're working on something (they hint that they may be planning to integrate indoor mapping into Google Maps which might be interesting) but they're not showing it in this video.

That doesn't mean the tech is bad. But we haven't seen enough to judge it as useful to end users.

Google has acknowledged that their vision of where this is going is, at best, partial: "While we may believe we know where this technology will take us, history suggests we that should be humble in our predictions. We are excited to see the effort take shape with each step forward."

I liked this part.

Google Maps already has indoor maps.

Yeah but they even said in the video that they struggle with indoor navigation and positioning.

This has applications even for, uh, "normal" people. For example, online real estate portals. People who are house-hunting would probably appreciate floorplans or a 3D tour. Agents, landlords or property developers would be able to easily create those with their mobile device.

>Agents, landlords or property developers would be able to easily create those with their mobile device

But they won't! It is very difficult to get those people to post more than a low res picture of one room online when they can very easily walk through the house with a video camera, for example. Even large apartment complexes have 1-2 pictures on their website and call it a day.

I'm currently house hunting (in Sweden) and most real estate agents go for full on visual overload. Everybody has at least 30+ 'artistic' photos. Most also have nice videos of the house on their site. Some even have clever 360 panorama thingies. I basically never watch them as they add nothing of value.

The single most important thing, for me, is a nice and correct 2D floor plan. Add to that 5-10 well chosen photos and you're done on the visual front as far as I'm concerned. If you haven't convinced to at least go look at the house with that, no video or interactive 3D model is going to change my mind.

The big problem is that the overlap between the information I consider important and the information the real estate agent is eager to share don't overlap all the much

Nail in the head.

I have chatted with one of the agents about how do they work, and no surprise - they lend out everything by phone, because "uploading pictures to website takes 24 hours."

Indeed. Panoramic photo VR tours were also gonna be the next big thing for real estate (remember QTVR?) but that only made sense for one-of-a-kind high end properties. Everything else is a commodity.

I used to work for one of the largest real estate portals in Europe, and people seldom watched the video of a property (if available). We also added floorplans from floorplanner.com and that converted way better. Now 25% of the listed properties has one.

To be honest, 25% is a very low number for such an essential piece of information about the property. Do you mind sharing how often people look at floor plans on your portal?

I'm a Russian living in the UK and it still surprises me that what Russians consider to be a viable information about property (gross internal and net internal area, kitchen area and a floor plan) is so rarely present on the UK property sites. "Lovely 2-bedroom" is all you normally expect.

That's because 99% of UK housing comes in a single standard size --- `XS`.

Do you see a lot of agents using Photosynth to give 3D tours ? It has been out for 4 years.

Sure with embedded in a smartphone would be more convenient, but I'm sure agents have some cameras when they get in houses.

I pretty much stopped reading when they were going through the possible applications. Find my way around a super-store, shit I'd rather just stay out if at all possible.

In the meantime, I'm sure this will revolutionize something. It always does. I'm just in your same position ... a neo-luddite.

Also, I would much rather have a space elevator than a 3D mapping phone.

The 3D mapping phone gets you the space elevator, because you need AI lab-assistants / engineers / scientists to build the space elevator.

It's not really an either/or situation

hah! That's the word I was looking for.

Since they want to "map the physical world", Google should buy the Euclideon guys already (Unlimited Detail Engine, Geoverse, etc):

https://www.youtube.com/watch?v=Irf-HJ4fBls

And then release it for VR applications/games (one can hope).

I thought they were investment-scamming techno-cons who had never been releasing video demonstrations for years without ever releasing a product.

The technology is amazing, but honestly these applications don't really seem that valuable, especially to the average user. Facial recognition is the main application of machine vision I can think of, but even that has limited utility. And I'm not sure if it will actually benefit from this technology.

What offices are in the second floor of that building across the street?

Is there something special about the house I'm pointing my phone at?

Who owns the daycare centre I'm pointing my phone at? Have there been any licensing / regulatory violations?

Paint a coloured path on the floor or wall to guide me to the dentist's office.

I'm old and can't see very well. Give me verbal directions to navigate (indoors) to get to the clinic.

I've got a wet spot on my basement ceiling. Highlight the outside of the house where water might be entering.

Any mechanical engineer who's had to take a 3D object and model it in CAD will love this.

I've been toying with the idea of such software in a phone for a while but never bothered exploring it because I lack the technical chops. There's a piece of software I've previously used called PhotoModeller [1] that allows you to calibrate a standard digital P&S camera then use a bunch of shots of the scene from various angles to build a 3D point cloud of it. Given that you can know the lens of an iPhone to a pretty close accuracy, I was thinking that you could build a similar application straight into the phone that then could upload 3d scans to dropbox. It'd be invaluable to field work.

This takes the above idea and loads it with steroids. I'm really excited!

[1] http://www.photomodeler.com/index.html

This is called photogrammetry - check out Autodesk's 123D Catch iPhone and web app, it yields some pretty good results if you take enough evenly lit photos with decent overlap.

Try this: http://seene.co/

It does just that and even let's you post the scenes.

It's a path to huds, in the Deepness in the Sky sense.

http://en.wikipedia.org/wiki/A_Deepness_in_the_Sky

You need situational awareness in a device to start using it to paint data onto the surroundings. Once you can do that, a whole world of applications unlocks.

Seems like this is much more likely Google's goal. Couching their arguments in ways the consumer will benefit left me feeling woefully unimpressed with the possibilities.

But clearly PK Dick's self-aware advertising will not be possible until such ads can distinguish a human from a column of marble or a dog.

This reminds me of William Hertling's 'AI Apocalypse' trilogy, specifically the last book 'The Last Firewall' where brain implants enabled depth-specific metadata info on everyone in vicinity.

It's machine vision. I mean the possibilities are endless. Your phone can see now. This gives the device way more information about an environment then a camera would. I can put this on the dashboard of my car or on the handle bars of bikes, I could leave this thing on at home while I'm work, I could use it to as a click counter at a theater to measure capacity, I could navigate pitch darkness much better, etc.

I think of it as similar in its final goal as at&t/olivetti research's "The Bat" ubiquitous or sentient computing project. These links provide some good examples of stuff they did with that: http://www.vs.inf.ethz.ch/events/dag2002/program/ws/Beresfor... http://research.microsoft.com/en-us/um/people/shodges/videos...

in short, the physical world can be treated as a user interface to computers.

There's a large technical distinction of course; in the case of the bat, getting the technology up and running to do this was sophisticated enough that the video i linked to spends a large portion explaining how it works. they used centralized computers and centralized sensors (echo-locators) to figure out where the device you are holding is within a known environment. Nowadays, we can put so much compute power and so many sensors in a device you are holding that it can figure out its own environment instead.

You don't understand it because it's beyond your comprehension, Morty. Because the world is full of idiots that don’t understand what’s important. And they’ll tear us apart, Morty. But if you stick with me, I’m gonna accomplish great things, Morty. And you’re gonna be part of them. And together, we’re gonna run around, Morty. We’re gonna- do all kinds of wonderful things, Morty.

It will let you see through walls.

Imagine being in a complex refinery, factory, etc., where tons of infrastructure is hidden behind walls or a couple of rooms away. Just hold your device in front of you, and peer through its virtual portal through the walls onto your hidden surroundings ...

Being able to know where everything is, along with an overlay of real-time status, etc. will be valuable.

> But what use cases does a consumer or the average phone user have for digital representations of the physical space around them, particularly given that the user is already aware of the the physical space around them?

For the user? Probably not much at the moment, if ever.

For a company whose mission is to collect, aggregate, and extract monetary value from every last piece of data in the world?

For a government that's interested in extending its awareness?

Priceless.

Example app:

You can see crowd moving on streets, where each person is a box (or a 3d avatar)... than if other people have installed the same app as you do.. it will show a icon on top of the box representing the person (the phone sends a signal IR, BT whatever) .. or is geolocated by a central server that sync the position of everybody..

Than you can interact with those strangers on the street.. in the 3d box you see representing the person, it may have more clues about that person..

so you can send a message to a girl/guy you liked and ask to hang out with you.. for intance.. its like a people radar..

Can work in traffic too.. so you can tag people in cars around you..

You can create a game, and involve people you have tagged, and give each one a role.. like in a RPG

If you get feedback of the camera too, you can see people with 3d stuff on top.. like holding a 3d gun..or a secret message in your virtual shirt.. it would work like a magical glass

I had this micro idea, a year ago.. this is the technology to make it work.. feel free to use it to create something

just call me for a beer later :)

the blind. that use case is even demonstrated in the video.

Ocular implants are not the newest technology. The argument I think would be that a phone would be a cheaper alternative/more accessible. But I'm interested if the 'audio cues' that they mention in the video would be any better than the audio cues already present in most environments.

I'm only aware of wildly experimental implants providing like 200 dots of light. They aren't by any means a deployed medical technology.

This sort of system could provide audio cues in any environment. That's already a big step up from 'most environments'.

Combined with something like Tactus [1] and the environment can be represented in a tactile 3d map [2].

[1] http://techcrunch.com/2014/01/10/new-tactus-case-concept-bri... [2] http://www.solidterrainmodeling.com/

Well I'd imagine using computer vision that can identify door handles, elevator buttons (even so far as reading out the numbers on the display).

Personal transport. It makes what Google did with self-driving cars possible with far less stable transporters: motos, bikes, even small crates with wheels or even rotors in large logistic centrals. It can mean plans like drug-delivery using drones have a better technology in crowded places, like when approaching futuristic health centers that synthesize or store drugs. This is huge.

Imagine if you integrate this with a future version of NEST thermostat or an integrated lighting, you can create different environments in a single room and change it on the fly. An electrician working in a Hotel would be navigated to the right junction box. You buy a new car and the manufacturer provides you with a virtual manual to find new controls, may be even basic repairs.

One example would be better versions of the iRobot cleaners. Currently they rely on a series of semi random movements and aren't particularly smart, which contributes to not being very efficient, and rarely getting into the edges of spaces. If you own one you've probably moved your furniture to accommodate their strange behaviour.

Neato's robot vacuum cleaners already do this using Lidar.

wow did you post this on the wrong website. "I don't understand"