I got to try this out when he was showing it in Toronto. It was a couple of months ago, so what I remember might not be exact.

- The demo included glowing hands of ice and fire. I did see my own hand with ice and fire glowing from the finger tips. I'm sure I saw it as blended around my fingers, rather than an overlay on top. - There was no latency at all, he let me type some stuff out on the virtual keyboard and the feedback was instant.

My only problem was it kept falling off my nose :)

Not true.

Looks like it's an Intel depth camera (http://goo.gl/ivrwY) glued to some Epson glasses. With the depth map from the camera, occluding virtual objects is possible.

Occluding virtual objects (real hands in front of virtual sphere) is possible. Occluding real objects (real hands behind virtual sphere) is not.

They're both possible and we'll have more videos very soon to show this. Filming through the glasses is really hard, we've been working on a rig the last few days and will have videos up soon.

Really? To occlude real objects, the glasses would need to have per-pixel adjustable transparency. They look like Epson Moverio glasses[1], and I don't see anything on Epson's site to suggest that they can selectively occlude light, but I'd be delighted to be wrong.

[1] http://www.epson.com/cgi-bin/Store/jsp/Product/Specification...

No, it does not support selective occlusion. Much like Google Glass it's a microprojector against your face, albeit this is two-eyed whereas Google's has one.

I think this is "possible" but unlikely to work well anytime in the near future. There are some really big problems that need to be addressed for it to work properly. To do this convincingly and effectively the glasses would need to incorporate eye tracking. Not only because the relative position of the screens is dependent on the way the glasses are resting on your head but most importantly because the placement of the screens in your view is dependent on the convergence of your eyes. If you're looking at something close (basically anything in the range of what you could manipulate with your hands) the relative position of the screens must be shifted closer together because otherwise you'll see double. It's really something that's very difficult to do and as far as I can see still at least half a decade off.

You're right. I misread your post and didn't realize they demo hands being 100% occluded by virtual objects. Can't believe they're that brazen.

Not understanding the objections. These glasses are a display, right? There's a computer in between ultimately responsible for what the eyes see. With enough clever processing of the scene, why wouldn't you be able to achieve this?

The glasses are not a normal display. Most displays are opaque, and can display pure black by simply not emitting any light. These glasses are transparent, so when they don't emit any light you see whatever's behind the display, not black. Displaying true black on a transparent display requires an extra element which can selectively block incoming light. These glasses don't have such an element, and cannot display black or dark colors unless you happen to be looking at a black object in the real world.

I don't see why it would be impossible; it'd just be pretty hard. What if you had another layer of glass that was like those old LCD calculator screens that have opaque numbers, but can otherwise be transparent? Maybe you could display things in front of those to have full opaqueness. That wouldn't help with variable opacity though.

You're right, an extra LCD layer could definitely do it. I'm just saying that the specific glasses being used here don't have this capability, unless there's some enhancement Meta isn't talking about and isn't showing in their videos.

Virtual objects occluding real objects is in fact nearly impossible. The way you're thinking about it with the extra LCD layer isn't actually a viable solution for this sort of display (or any sort of display my mind can come up with right now). The way glasses such as these work is a lcd screen hidden in the side of the glasses is viewed through a beamsplitter that combines the image of the screen with the real world. The screen appears to be "at infinity" from the viewers perspective. Unfortunately to do occlusion on glasses like this the light would have to be blocked between the source of the light (outside world) and the viewers eyes. The light blocking layer would always appear blurry from the users perspective since it would be so close to the eyes. You would also have problems aligning the light blocking layer with the screen as the apparent position of the screen in the glasses is dependent on the position of the glasses on your head.

Wow, you're totally right! Alignment problems are solvable with accurate positional tracking at least in theory, but I hadn't considered the blurriness problem. That seems like a show-stopper. It might be fixable if the light-blocking element was essentially a holographic display, capable of treating light differently depending on the incident angle, but such technology is going to be out of reach for a long, long time.

There is some nascent technology that allows the eye to converge on two focal planes simultaneously, which looks like it can solve the bluriness problem[1], but I don't get the impression that meta is using this. And it doesn't solve the issue of ensuring parallax alignment between the display screen and the background, which can only really be done if you're tracking the pupil and compensating accordingly...

[1]: http://www.bbc.co.uk/news/technology-17692256

To be fair, it looks like all the demos that show a person actually wearing the glasses have slightly translucent objects. While it is technically impossible to occlude an object 100% with this technology, I believe the percieved opacity of the augmented reality object is based on the difference between the intensity of the projected image and the real world background.

Theoretically, it should be possible to overpower the background light and have an object appear to be totally opaque, which is similar to the principle behind one-way mirrors. That amount of light might be blinding in certain circumstances, though.

Theoretically you could render occluding white objects that are strictly brighter than anything in the real scene by overpowering the background light. Dark objects, shadows, or strongly colored objects could not be rendered in this way, as they require blocking incoming light of the wrong brightness or color.

In practice, overpowering background light is not possible with current displays. They're simply not bright enough.

That is a really good point. I seem to remember a really interesting article posted to HN awhile ago that went over all the pitfalls in depth. I'm really wishing I had bookmarked it now.

I think I'm going to have to rewatch the promo video in depth to see if they have any unrealistic demos with glasses-wearing users.

I still believe this technology would be really useful for doing Real Work. You could get around the shadow problem by working in a black surface. I am imagining an interface with a black drafting desk, small bluetooth keyboard, and a 3d tracked pen with this AR display could be absolutely revolutionary for certain engineering disciplines.

Agreed. 20% of what they're showing might work, the rest is nice Hollywood VFX.

> The videos there are misleading.

I was thinking the same just based on the repeated "there is nothing like this being developed", which I may have believed if I didn't come across Oculus Rift just a few hours prior.

I think that claim is actually accurate. The Oculus Rift is much different: it's virtual reality, whereas Meta is augmented reality. They are different things, and augmented reality is much harder. The Oculus Rift completely occludes your entire field of vision and replaces it with something else entirely. Meta modifies the real world with virtual objects.

Couldn't one mount a camera ontop of the Oculus Rift (just like with Meta) and stream the camera feed into the Rift + process the video stream to offer augmented reality?

This would work much better. The main reason is latency. The added image will always lag behind the real world by some milliseconds (16ms minimum refresh latency for 60hz). There is also the image capture and processing latency which is considerable. This is true for both the Meta (hard augmented reality - hard AR) and the occulus rift (virtual reality - VR).

The difference in experience is for VR, the whole world lags so an added object will always be at the right place. With hard AR, the added object will be trying to catch up as the image moves.

From what I gather on google glass, they mostly aim for soft AR. This is where you add information on top of the real world but it doesn't map directly to a location in the real world: if you move your head, the added image stays the same. This is much more resistant to delays and could be useable for extended periods of time without discomfort.

For a better and more technical writeup:

http://blogs.valvesoftware.com/abrash/why-you-wont-see-hard-...

> the repeated "there is nothing like this being developed"

Maybe it's meant to include themselves?

To support parent, a somewhat technical writeup:

http://blogs.valvesoftware.com/abrash/why-you-wont-see-hard-...