> Take an empty game (update nothing, draw nothing) and send black over the wire. You blow the budget.

You say that, but as a user I wouldn’t notice the difference in this game played on cloud versus locally.

Speak for yourself. I've tried every iteration of cloud gaming and the frame lag is painfully obvious to me. Maybe it's because I've been gaming on a PC for 28 years.

This dichotomy of player opinions seems to be present any time I see a discussion about Stadia or cloud gaming. I haven't tried it myself so I don't have a truly informed opinion, but just based on technical factors I'm extremely skeptical.

However there does seem to be legitimate portion of the audience that either can't tell or simply doesn't care about frame lag/latency issues. I would guess that group is relatively small though. Also as PC and console hardware continues to improve and 4k+/144hz+ gaming becomes more commonplace, cloud gaming is just going to be left in the dust.

Whenever I've advised VCs on cloud gaming tech, I always point out that you can have the beefiest most expensive hardware at the edge, but you'll never be faster than an HDMI/VGA/RCA cable. It's just physics.

Cloud gaming will work great when we have FTL transmission of data or possibly a datacenter in every 7-11/Walmart.

Well, they did speak for themself.

Absolutely. There are certainly classes of games where several frames of input lag don't matter, but look at the big AAA titles, look at the popularity of challenging action games like Elden Ring, of multiplayer shooters and MOBAs.

It is, at best, a niche service.

Isn't Elden Ring the kind of game where you start an animated attack then basically wait for it to finish? So the game locks in your moves? Meaning some lag may be less noticeable?

It locks you after you start the animation, but the key here is when you start the animation.

If you see someone is going to attack you, you want to dodge right away, wait for the animation to finish and attack after. With cloud gaming you see someone is going to attack you, press dodge, get attacked, you die.

Unless you compensate. Receiveing a button press means the button was pressed <latency> ago. Rewind simulation to that moment, apply input to game state, replay to current moment, send result.

Of course, single-player games don't usually do this, because it's very complicated.

You are sort of comparing apples to oranges. Like consider a beefy gaming computer, but the mouse is some junk with an input latency of 10ms. Can this computer do 144fps? It most certainly can. Despite the input latency, 144 frames are rendered every second, and 144 frames are displayed on the monitor every second. Movement and animations will be butter smooth. For the player the latency is frustrating but consider the friend looking over his shoulder. He can't tell. All he knows is that there are in fact 144 frames per second.

So you gotta ask yourself, well when does this make sense? Does it make sense in a twitch shooter? It doesn't. Does it make sense in a popular mmorpg that runs with a tickrate of 600ms? Yes. Does it make sense in a RTS? Probably, except at maybe the highest levels. Does it make sense in a train simulator? Yes.

I think it works if you do it over a generation (of people). Never give a kid less than 50ms of latency and they won’t even realize they’re missing out.

The whole industry will attack the goal from multiple angles and make it happen. I think making GPU ownership expensive to improve the value proposition of streaming services is one of the goals.

On paper I think it should be a total failure. In reality, I tried the RTX 3080 tier of GeForce Now and I couldn’t notice the input latency playing Last Epoch. If triple buffering was accepted, 3 frames of latency for streaming will be too, won’t it?

I hope it fails because never owning anything sucks, but I think it’ll be extremely successful.

Triple buffering does not inherently introduce any latency. It's a technique of using more memory to smooth out synchronization issues.

Single buffering: You're drawing pixels directly on screen. Problem: Drawing isn't instant across whole screen, so human eye sees a blend of old and new image (and since screens draw top to bottom, this produces a horizontal "tear" somewhere, thus "screen tearing").

Double buffering: You're drawing pixels on a hidden screen while the previous image is displayed, then flip screens. This flip action is so fast it's practically instant, so tearing is solved. However, screens can only flip at set intervals (refresh rate). Problem: What if your drawing finishes even a little bit late, just after refresh rate sync? Now your program has to wait for the next opportunity to flip, which just makes everything hang.

Triple buffering: 3 is the magic number to fix all these problems. You always have two virtual screens to flip between, and a third one you can draw to regardless of flipping.

But isn’t the drawn screen always two frames (32 ms) behind the game loop’s current frame?

No.

If the game renders on time, what you see on screen is always the latest completed image that was ready at previous vsync.

With two buffers, if you render too long and miss vsync and then call flip() which waits until the next vsync, you're wasting the frame waiting (because the system can only swap buffers at vsync, to prevent tearing).

The third buffer only exists so that the game can start working on the next frame, while two of the buffers are "locked" (one on screen, other ready for display, third available).

Latency perception is highly game dependent.

I worked on the netcode for EA games at one point in my life.

The highly popular battle royal and competitive shooters will never go cloud.

>Never give a kid less than 50ms of latency and they won’t even realize they’re missing out.

The majority of the people now think 24 frames a second looks natural and 60fps looks fake. They're twisted by their malevolent environment into a wrong and unnatural state, like elves into orcs.

Well, that may be true with having to send the entire payload over a distance of space-time. However, if AI-based rendering could sufficiently predict the next frame, then you can just simulate the next frame without lag. Given it can predict enough frames in the future in relation to compute requirements, then all renders are local-only with "online synchronization" - at least, this will probably be how we do long-distance "calling" when we are a multi-solar system species; unless we find a way to break the speed of light limit. In a way everyone you talk to really would be a simulation; and they'd "sync" up to new data every so often for alignment.

> However, if AI-based rendering could sufficiently predict the next frame, then you can just simulate the next frame without lag.

How would you predict / simulate when an opponent jumps or crouches? And how much processing power would AI rendering take? I don’t know anything about AI.

I think the idea is to run the client side on a potato.

I'm not convinced that 16ms or 6ms is the right budget to talk about. Sure, if you want someone to see the picture, make a decision and reflect it back in the next picture, that would be the budget. But take a comparison to telephony, the ITU says the budget for unnoticeable latency is 150ms. So I can talk to you on the phone with 150ms latency and not even notice. Now, I think gaming is probably lower than 150 because the timing of speech is fairly slow compared to twitch shooters, but I don't think it's 6ms. You might be right it's still unachievable (and I agree, the only viable solution might be to boil the ocean by sticking data centres on the end of every street and create a new guaranteed low latency protocol that takes precedence over all other TCP traffic), but I'm not as certain.

Don't forget about the baseline input latency that even local games experience. The best case scenario is PC, which would add in the realm of 10-20ms of input latency to your network latency. TVs (which are the target for this stuff) have hilariously large input latency.

You're teetering on the budget before you even hit the network.