Even if you had no issues in rendering latency, you're overlooking the data aspect. Any open-world AR, is going to be continuously downloading and discovering new geometry, models, textures, mobile programs, as you walk around.

Let's say you walk up to a restaurant. As you look at it, you see a virtual waitress outside. The you can see the menu floating there, interact with virtual food models, interact with a fully animated and articulated intelligent agent. This all has to load dynamically in a an "AR Web"-like environment (no way you're going to download and install an APP per local brick and mortar).

For this to scale, and not have hideous load times, there's got to be some kind of quad-tree/oct-tree like index (or other spatial acceleration structure) that's streams in, LOD streaming, and for this to work, the "sidelinking" stuff they talk about will be a huge boon.

Because the "internet of things" in an AR world isn't an "internet of PHYSICAL things" like door locks and web cams, but also an "internet of VIRTUAL things" to control, like virtual vending machines to order food. That means somewhere nearby, there's a next-gen router-like device, that offers side-linking, and has an IP service in it serving up assets to you. Now, it could go up through your phone carrier, through the internet backbone, into a cloud provider, or it could avoid that round trip and talk directly to the restaurant's "virtual POS" 5g device.

Point is, we don't know how this is all going to work yet, and it's very premature for people to pooh pooh the use cases for higher bandwidth or lower latency.

"Let's say you walk up to a restaurant. As you look at it, you see a virtual waitress outside. The you can see the menu floating there, interact with virtual food models, interact with a fully animated and articulated intelligent agent. This all has to load dynamically in a an "AR Web"-like environment (no way you're going to download and install an APP per local brick and mortar)."

Maybe I'm just getting old, but this all sounds tedious and cumbersome compared to just having a menu stuck to the front window.

I think this has been borne out for most network technologies: the fundamental benefit is not {human doing old thing over new network} but {machine agent doing old thing for human over new network}.

But the genesis is always a bit counter-intuitive, because people don't build things for which there's no use case.

E.g. Machine agent consumable mobile services were created because people had mobile devices capable of consuming them. Nobody would have wasted time porting physical data into machine-readable format before the devices / use cases existed.

So the killer app, in this case, would be having your phone auto-filter / -alert you if for menu item(s) that match your preferences. Or create an ecosystem where menu costs are negligible, and restaurants are free to pivot to ingredients-of-the-day.

Carmack famously quipped that "I can send an IP packet to Europe faster than I can send a pixel to the screen" [1]. For VR and AR to break through, they're going to have to be in a form factor that resembles normal glasses, and at a much higher render quality (4K+) than anything we've got now. That's almost certainly going to require rendering remotely Stadia / Xbox Cloud style, with some minimal local processing.

[1] https://superuser.com/questions/419070/transatlantic-ping-fa....

Well, or you could render on the phone and stream to the glasses.

I don’t think even 5G is low latency enough for what you’re describing—not even close. Stadia works because it’s not VR, so they have some latency headroom. For good VR (Valve Index, etc), you get around 20ms for the entire rendering chain.

Depending on bandwidth, you could probably stream an over-rendered viewport (e.g. 3D skybox) that allows for local-device-magnitude latency for critical functions (perspective change & location translation), while being backed by streaming servers (especially of the edge variety).

I suppose in the future, anything is possible—but I don't see it. You're asking the local device to do Z translations of a 3D scene when all it has is an already-rendered 2D image.

And the local device will just have a 2D image. Even within a single, local PC, multi-GPU rendering is basically dead for video games because no one has been able to get them working reliably. And that is with the full bandwidth and latency of PCIe.

I don’t know. As long as they can render in 4K the actual geometry can be fairly simple, so maybe my phone would be perfectly capable of that?

Remember a frame at 144hz is still 7ms - assuming everything else in your pipeline doesn't add up to much, 1-2ms added from the network should be fine for a good VR experience as long as it's 99.99...% stable. If the latency is constantly changing (as it tends to do with wireless networks) it's going to be much more noticeable.

Graphics hardware keeps getting more powerful and cheaper though, I don't think cloud rendering is going to get more useful over time compared to just rendering on the phone and using cloud streaming for big assets.