What do you think will be the keys to any future battle between this, something like Apple Vision Pro, and traditional desktop metaphors? I have to admit, on first viewing every single component of this looks unbearably clunky but presumably it’s going somewhere, and I can’t say I want to live in a future where I have to strap on a headset either.

I have become convinced that the only mass-marketable form of spatial computing is a neural implant or contact lens.

The sunglasses form factor would be more mass-marketable than contact lenses, I believe.

My only concern is for people like me who already have really poor eyesight (mine is -9 on one and -11 on the other). I am envious of people who can wear sunglasses, and the only option I have is to get contacts and that doesn’t seem very pleasant. Also why I never fully came around to VR/AR because it felt like I’m wearing double goggles, but I acknowledge that there are others who wear glasses and don’t share the same concerns.

What does the poor eyesight have to do with not being able to wear sunglasses?

There are prescription sunglasses for shortsightness, astigmatism, etc.

So the sunglasses-form VR would just have to cater for that. Apple's headset-form VR already does (offers the option to have prescription lenses in it).

Or just find a pair that fits over the prescription pair and wear them both.

Not easy to wear two pairs at the same time, and not convenient either.

They probably meant clip-on sunglasses that clip to your usual glases.

no, I do mean just wearing two pairs of glasses, one on top of each other.

With extreme corrections it get harder to put that into anything other than flat lenses and a lot of sun glasses aren't that shape. I recently went through getting perscription safety glasses and had to get special inserts to flatten out the lens profile to make my relatively minor -3.75 to -4 prescription. I was able to get some relatively normal Oakleys in my script for sunglasses though.

Also you generally want sunglasses to be much wider than prescription glasses, because otherwise you get too much light bleed around the sides. Making large lenses is when the thickness takes off, especially for nearsightedness where the thickness needs to be at the edge of the lens.

I would expect the second / third generation of sunglasses form would be showing you a high quality video display instead of seeing "through" it - so I suppose correcting for vision problems would be zooming or other effects on the video. This of course is my expectation based on no knowledge in the field and probably sounds silly to those with that knowledge.

Don't worry. I'm blind and I'm totally fucked in a 3d first ux. :)

I used to absolutely dead contact lenses and not understand how anyone could choose to wear them. Then I got diagnosed with keratoconus and had to wear rigid contact lenses. It turns out it's not that bad, especially with hybrid or sclerals. I imagine entirely soft ones are even more pleasant if that's an option for you. It's gonna be weird the first few hours or and it will take a few days to get comfortable handling them, but it's really convenient. You don't have to clean them throughout the day like glasses, you can wear sunglasses over them etc. I recommend you try them for a few days and see how you actually like it. It's not a scary as it sounds.

I've put lenses on all my sunglasses. Works great, except when I go into a store and I have to lift my sunglasses and suddenly it's all blurry.

I have a matched set of -9, so I don't really have to track left/right with contacts. Which is kind of nice.

Day to day, I wear glasses, and have no plans to change that. If I'm going to a show or a movie, or if I'm planning on spending a lot of time outside, I wear the contacts. The contacts are set up for distance not reading. and that's becoming more important as I get older.

It can be disorienting for a few minutes, I forget how much glasses distort peripheral vision.

I get a small supply of daily disposables. I keep a pair in my laptop bag. With vision as poor as ours, you know how difficult it is if your glasses are lost or damaged. It's not life changing, but it is a genuine quality of life improvement for me.

I encourage you to get a contact exam, to verify you can wear contacts. They should teach you how to put them in and take them out. It can be stressful, but you never touch your eye. The doctor or assistant can answer any questions you have. They're not particularly expensive (in the US anyway). You don't have to wear them. Try them out, they might have some benefits you haven't realized.

That said, I don't think i'd wear contacts just for AR/VR. The headsets already seem like a hassle, and extra tooling to make my eyes use them effectively, even more of a hassle.

A lot of online glasses places do two-for-one deals so I just get two of the same frames, one with normal lenses and one with tinted. That way I know the lenses will fit the frame.

Is it the idea of contacts that you don’t like? My wife is in the -15 to -16 range with astigmatism and has worn contacts for 25 years. She doesn’t have any abnormal trouble with VR when wearing them. At -11 you can still get glasses lenses at all the budget places, 1.74 refractive index even!

There are sunglasses (normally worn by older people) that are meant to be worn over your normal glasses.

Feels like that's the endgame, even if it's decades away. Being able to arbitrarily address a human's supposed 576-megapixel visual resolution trumps basically anything else. Don't think this is relevant for this next generation of UIs though (if that generation does indeed materialise).

The numbers I’m familiar with are: 120M rods and 6M cones (each eye). What’s the derivation for your number? Is it projecting the rod density at the fovea over the whole visual field?

Ahem, my calculation was that I once googled it and remember the number. My point’s more about having complete control of your vision rather than the specific number, but I assume that will one day be strictly better than any possible screen.

from: https://clarkvision.com/articles/eye-resolution.html

How many megapixels equivalent does the eye have?

The eye is not a single frame snapshot camera. It is more like a video stream. The eye moves rapidly in small angular amounts and continually updates the image in one's brain to "paint" the detail. We also have two eyes, and our brains combine the signals to increase the resolution further. We also typically move our eyes around the scene to gather more information. Because of these factors, the eye plus brain assembles a higher resolution image than possible with the number of photoreceptors in the retina. So the megapixel equivalent numbers below refer to the spatial detail in an image that would be required to show what the human eye could see when you view a scene.

Based on the above data for the resolution of the human eye, let's try a "small" example first. Consider a view in front of you that is 90 degrees by 90 degrees, like looking through an open window at a scene. The number of pixels would be 90 degrees * 60 arc-minutes/degree * 1/0.3 * 90 * 60 * 1/0.3 = 324,000,000 pixels (324 megapixels). At any one moment, you actually do not perceive that many pixels, but your eye moves around the scene to see all the detail you want. But the human eye really sees a larger field of view, close to 180 degrees. Let's be conservative and use 120 degrees for the field of view. Then we would see 120 * 120 * 60 * 60 / (0.3 * 0.3) = 576 megapixels. The full angle of human vision would require even more megapixels. This kind of image detail requires A large format camera to record.

all your resolution is clumped up in the middle (around the fovea), and generally, nobody treats vision as being a rectangular grid of pixels (for many reasons).

First problem: power and connectivity - you can't have wires dangling from inside your skull or eyes.

Second, that _completely_ ignores touch, which is the largest of the senses. There's a reason car companies are going back to knobs!

Third problem, all devices produce heat. Lasers, LEDs, and cameras. There's a hard limit to how much heat you want to conduct into the cornea.

Fourth problem: it will all be used for advertising, surveillance and disinformation anyway. Beamed directly into your skull and eyes so you can't avoid it? Awesome.

Going to be great to have to get a neural implant to get a job.

The tree of Spatial Computing has many branches. The use of Tangible User Interface mechanics in a super specific application like this is a perfect bottom up design from function approach and I love it. He has all the building blocks, no pun intended, to reshape the same experience deeply customized for any number of domains like digital twin monitoring of manufacturing plant lines or dynamic ERP dashboards or K-12 like the original Dynamic Land work. Looking forward to whatever he is getting ready to reveal.

This looks super impressive, but at the same time, I'd really like to hear someone actually working in molecular biology what they think of this. While I am working in this field, I only deal with software, not molecular design and wetlab stuff. But I see how people there currently work and have a very hard time imagining them fiddling with pieces of paper and origami blocks on a desk, looking at poor quality images and awkwardly move stuff with a laser pointer. I like the physicality, but at the same time, it seems very limiting to me in terms of scope.

This is very impressive work and I'm glad people are doing it. For any billionaire, funding it would be a no-brainer.

That said, it makes more salient for me the dichotomy between demos and reality. This demo looks very much like molecular biology does in the movies.

But what is it to look like something "in the movies"? I don't know exactly... All tools have overhead, no matter how cleverly designed. Maybe "the movies" is what you get when you ignore this overhead.

There's a clear phenomenon in looking back on old demos and concluding we must have taken a wrong turn because our systems are still not as good. This is true of Victor's own demos from a decade ago, and many well-known demos from decades earlier.

But rewatching a Victor demo earlier this year, which had blown my mind when I saw it live, it seemed full of a kind of naivety (and a bit of sleight of hand). Of course, he made clear it was only a demo. It was meant to inspire. But now I feel that's an easy excuse for not investigating what really advances our use of tools – and what holds it back.

You can just say “Worse Is Better”.

> …what really advances our use of tools - and what holds us back.

It’s money! It’s cost! The cheaper tool wins.

“Worse Is Better”

> For any billionaire, funding it would be a no-brainer.

Billionaires still want ROI.

I have no idea what I just watched. :-( Bret Victor makes me feel inadequate.