My understanding is that occlusion in "see through" AR is an unsolved problem, everything looks ghostly and somewhat semitransparent. Until someone solves that I suspect the re-projection method is the only viable option.

Yes "projecting black" is effectively impossible - every once in a while someone from Magic Leap would claim they had solved it but everyone knew it was bullshit

I'm sorta surprised by this. Don't liquid crystal displays in laptops do exactly that?: LCDs are a controllably transparent array that is overlaid on a uniform backlight. So why can't one create a set of glasses where the lens are an LCD array without the backlight?

An LCD simply turns "off" to display "black" - so the emissions stop (relative to surrounding pixel emissions)

With a projector, you can't "throw" nothing (aka black). As a result "projected black" is simply lack of projection.

In the case of a translucent or transparent reflection or waveguide surface - which is what the projection reflects off of - "black" is whatever the darkest part of the surface is. In effect whatever else is emitting from the surface that you're looking at will change the depth of "black" you get.

This is why the Hololens and other see through AR devices are always tinted, to set a higher threshold for "black" than the surrounding unaided view.

It's more accurate to say than an LCD allows light to pass through a 100% polarizing filter. The are three layers of polarizing material. The two outer layers are at right-angle polarizations to each other and normally would be completely opaque on their own. When power is applied to the liquid crystal, it twists the crystal's polarization to be at a 45° angle to the other two layers, which then permits about 25% of any incident light to pass through.

An optically transparent waveguide display can use an LCD layer to block light coming through the front and then not render graphics on that area of the display. It will be opaque black at that point (though rather fuzzy around the edges, as the LCD won't be in focus).

Magic Leap 2 actually employs this technique. It's... a lot like the rest of the device: a good idea on paper.

That’s a fair response, though I hope you’d agree that in the context of discussing pass through Vs see through “black” the majority of use cases are indeed fully occluding/lit LCDs near eye and not ML style lenses.

But you were talking about the Magic Leap, replying to someone talking about waveguides...

Magic Leap.. How are they still around? And have a valuation in the billions?

I don't know. I mean, their newest device is better than the HoloLens 2. But like, that's just a relative statement. Waveguide displays are still objectively dogshit.

HoloLens stopped being a focus of Microsoft and it shows in the product

https://tecl.ink/2023/06/apples-vision-pro-wearable-headset-...

You have to be looking through an LCD for that to work, but all existing AR HMDs are head mounted projectors with transparent mirrors, with no means to control, say, pixel level local reflectivity of the mirror. The mirror is just a flat mirror.

If you forgo the projector part and replace the mirror with a transparent LCD, it’s just too close to your eyes and you can’t see anything. If you add a microlens array to the LCD, now the LCD might come to focus but background becomes way too far-focused, and you can’t see anything either.

If we were on an Enterprise-D, I guess I could just ask replicator for a passive illuminated metamaterial light field image combiner with integrated processing than runs on bus power from DisplayPort input, but we are not there yet.

So, for now, our AR HMDs can only brighten pixels against backgrounds.

I think I understand. The intended idea was, e.g, a pair of "glasses" where in front of each eye are two layers, (1) a partial transparent mirror and (2) an LCD array, so that you could darken a pixel to block light coming through the mirror, and you would ideally only see light from the projector that was being bounced off the mirror and entering your eye. But you're pointing out that those LCD pixels won't be at the same focal depth as the objects out in the world you're looking at. Thank you.

This is a concise way of describing the problem I think

I work in the space. Occlusion only matters if it matters. "Ghostly" information display is suitable for a shockingly wide range of tasks.

While that's true, I believe these (mass market) devices need to be general purpose, and so need to cover the use cases where occlusion is necessary.

This is also what I intuited. Can you give some examples of things that require “projected black” , especially those that couldn’t be solved by using a darkened room?

Right, I think it's going to come eventually but it's really really really hard

They are still asking suppliers to solve the technology, yes.

Hopefully we are not forgetting about Bosch's Retinal Projection which solves many problems with lensing HMD AR. If I was a betting man I would say Bosch came up with this tech almost specifically for Apple to integrate it into interactive glasses.

Yes...

https://tecl.ink/2023/06/apples-vision-pro-wearable-headset-...