That's real-time rendering though, right? Is there anything preventing it from being pre-rendered in non-real-time first? Or does it have to be rendered in real-time?

I'm not familiar with any of this at all, so I'm genuinely curious.

If the context was to use in a VR/AR headset it has to be real-time. And I guess that use-case, and the related that you interactively want to walk around a scene are two of the main use-cases

I think the best way to consider it.. is as a 3-d cloud of individually addressable pixels. The size of the cloud is dimensions of real-time rendering.

The interesting bit is that it's feasible to render in realtime. We have plenty of ways to render a 3D scene if render time is not an issue.

I thought they recorded their special videos using the Vision Pro itself, which has enough sensors to build depth maps of the scene and provide novel views within a small range from the original position.

But I am half speculating and I don't really remember. That's just the impression I remember having.

That’s a feature of the headset, but I think some of the demo videos were recorded in some other manor. I seem to remember hearing a discussion on a podcast (The Talk Show? Dithering?) where it was mentioned you could see a camera rig somehow.

Perhaps that was something you could see on TV when they accidentally showed the rig that was at the same event?

Of course it’s possible they were only using a camera rig so that no one would see the device before they were ready to invade it. Which would be very Apple like.

So I’m speculating a bit. But even if they were going to professionally record events I would think they would do better than just have someone sit there with a headset on.

I saw the image you’re referencing yesterday. The unit appeared to be a plane, briefcase sized. It had two fisheye lenses about 10cm diameter each and separated by what I expect is an average human IPD.

So I think that unit was simply capturing 180 degree stereo video. Not enough to compute volume without most of it being some ML inference.

It’s not full volumetric video. Probably can simulate a few degrees off axis but nothing like what this article shows.