"Testing has found that overall "input lag" (from controller input to display response) times of approximately 200 ms are distracting to the user. It also appears that (excluding the monitor/television display lag) 133 ms is an average response time and the most sensitive games (fighting games, first person shooters and rhythm games) achieve response times of 67 ms (excluding display lag)."
These numbers are from the PS3/XBox 360 area, which certainly sold massively well. Since display lag is not included in these 67ms, this is what these streaming services would have to aim for. It doesn't sound completely insane given the 30ms round trip latencies of home connections.
On a more subjective note, I think it's going to be really hard hunting down high end gamer PCs. On the other hand lots of games don't run particularly smoothly on consoles (especially "normal" PS4 and XBox One, rather than PS4 Pro or XBox One S) that these services are directly aiming at. I can perfectly imagine a stadia version of The Witcher 3 running circles around the PS4 version that lags so much... But I don't think it would hold against the PC version.
But I don't game much at all anymore, so my knowledge is mostly outdated. The last games I played were 80 days and A Dark Room on my iPhone (both pretty good games). So I guess I'm now more in the target demographics of Apple Arcade.
Personally I noticed a significant improvement switching to a 165hz monitor in basically every first person game. The only difference here is going from 16ms to 6ms frame granularity (10ms off the worst case). I’m not sure how much is visual smoothness or latency, but I’d gladly take refresh rate/fps over UHD.
I’m definitely in the enthusiast group though. I didn’t even realize that console players dealt with such high latency. It will be very interesting to see how the market plays out.
I am not totally convinced that a high refresh rate monitor allows human vision to respond faster. I would rather suspect that most game loops are VSync-locked and a higher refresh rate leads to better input sampling and processing.
Maybe I should set up a simple test where I show a non-interactive sequence at various framerates and ask users to identify the highest framerate. And then repeat with interactive camera controls.
Have you ever watched a high-level FPS player? They can aim and shoot in the span of about ~1/5 of a second, occasionally even faster. That's 200ms. At 25hz, that's 40ms. A big part of it is going to be increased input processing, but a lot of it is muscle memory at that skill level. At 165hz, that's 6ms. With two equally skilled players, one will see the other up to a whole 34 ms sooner, in a process that takes about 200ms total. That's going to show up as a highly statistically significant advantage. Of course, the increased sense of connectedness to the game at higher rates is another big advantage, but these effects combine. Pit one skilled shooter against another at equal refresh rates, but with one dealing with 34ms of additional input latency, and they're not going to play as well as they should.
Competitive players explicitly disable vsync, because the higher the game framerate, the faster it processes input. Also, the sooner you see updated information on the screen mid-refresh, but that doesn't help as much as a wholly higher refresh rate.
All of this only matters if cloud gamers were playing against locally-rendered players in the same match though.
This is not reliable at all. You can really feel 100ms latency.
That's why in the world of pro music, barely anything above 10ms keypress-to-sound latency is deemed acceptable.
Quake III will make all these streamed games feel slow.
Quake III makes all modern games feel slow. But that hasn't had the slightest effect on the typical gamers mindset. If only certain type of games work with cloud gaming then AAA publishers will only make those type of games. Exactly like the case with EA and microtransactions.
I can see this working very well for games like Total War where hardware specs are super high but response time is not critical
However, with these streaming systems I’m not sure developers will be able to use tricks like that. It seems like these tech stacks are running all code remotely. I feel pretty skeptical about whether they can make the game feel good enough this way.
However that's not most of the market. The appeal of cloud gaming is for single player non-competitive games. These are the most pirated games and the games where input lag matters the least.
Sure you won't get high end PC gamers, that's not really the target demographic here. It's the millions of console and mobile gamers.
Moreover I was able to play a AAA title on my 2011 Macbook Air and that was fantastic.
This is actually already done in many AAA titles. The idea is to basically always be showing a prediction of what will have happened in ~70 ms, and update your state model accordingly. Given a datacenter to run the games on and all of the announcements about being able to take a snapshot of the state, this doesn't sound super far-fetched.
On a streamed game, all interactions need a round trip before the user sees any response.
Not to mention that the game will have to be written especially to allow for the simulation to be rolled back and forth. For example a misprediction triggers a sword swing it turns out you didn’t make that traps the player in a state for a second. You end up giving the player a horrible experience where the game seemingly has a mind of its own. Or you can roll back and interpolate from the bad state back to a good one which would be ideal. Then the middle ground is to rollback and have a cut in the action back from misprediction. Then you realise this is happening every second or two.
It's hugely dependent on particular network conditions. I wonder if SpaceX Starlink might have an option to optimize for latency?
This is why I'm suggesting you try it. Because to be more blunt, you do not know what you are talking about.
It's a different story on a pc with a decent gaming monitor though.
If you have a 100ms latency to a stadia server, then there's a 200ms latency between pushing forward on the stick and your character moving forward. This is not the case on a networked game, the client would start moving your player immediately (even if there is a 100ms delay to send those updates to the server).
It's also easy to port edge-rendered games to many platforms. It makes sense for MS to ally with Sony and shift their effort to cloud before both the Xbox and PlayStation are little more than rich web clients.
CDNs are deploying compute/lambda at the edge capabilities, so "gaming at the edge" could be seen as a reasonable expansion of that.
There will have to be local rendering for nearby objects and particles, but this is feasible.
DRM isn't totally likely, REnouveau does exist, and basic reverse engineering tools could be created to document the most common game engine functions.
Twitch shooters will never be replaced by this though.
However much lower latency to these services is definitely viable in a lot of places as long as they put enough servers spread out across the country.
In the USA at least there's going to be a lot of areas where the available options for internet are just not good enough though.
I think it's actually more or less the opposite?
Twitch platformers like Celeste and Cuphead can't afford input lag, as they require very quick and precise movements from players. It so happens that these types of games almost always have low system requirements. Rhythm games and fighting games tend to be similarly lightweight. Street Fighter 5 won't run on a potato, but the hardware required is far from state-of-the-art.
By contrast, Assassin's Creed Odyssey and Red Dead Redemption 2 already contain large amounts of input lag when played locally according to Digital Foundry, and consumers don't seem to care much. This isn't to say that adding more latency is a good idea—and I have no idea if the existing latency could be whittled down to compensate for streaming—but it clearly doesn't bother people much.
I'm working on a system that unifies single player and multi-player development. Think of it like zeit.co's "NodeJs Now" but for games, with a local-only configuration option. The way my system relates with streaming, is that the client is scaled way down in complexity. Basically, the sync protocol is a stream of positional updates from the server. The client becomes little more than a screen entity display. Then, the developer has the option to add a bit of "own world simulation" to further hide latency. (Or, in the case of the game I've implemented, just "own entity simulation" is enough.)
If you look at things like the Steam hardware survey, it becomes quickly apparent that the hardcore gamer (e.g. someone who has a high refresh rate display or similar and cares about this) appears likely a very small part of the overall gaming landscape, which is a very broad church now.
I think game streaming will ultimately win out hugely - “ordinary” (for lack of better term) gamers simply don’t care about the technical disadvantages. Huge numbers of console gamers already often add 100ms or more lag via their TV’s slow image processing and don’t even notice or care. I don’t think the average gamer is as latency sensitive as many readers here might be.
Most people are going to see that they can have a high end console/PC-like experience for a fraction of the cost of owning a high end console/PC, for the vast majority that is surely an appealing prospect. Finally, John Carmack himself believes the latency issue can be mitigated well enough for most things - that’s good enough for me ;)
> https://twitter.com/id_aa_carmack/status/1108144741932249088...
My only real concern in all this is the future of video game mods. This is a pretty big thing now and hard to imagine this surviving well in a streaming based future. This would be a loss, creatively and functionally, for some people.
At least in the UK a typical home broadband latency is 6 to 9ms in my experience.
SO COOL!!
How do they do it? By defining high speed as dial-up speeds in the ordinance they helped the local small, technologically inept government craft!
God Bless The USA
> Latency insensitive games for the most part are not difficult to run on commodity hardware.
Majority of people don't have a PC with a dedicated GPU. Or PC at all. Thanks to the Smartphone.
That said, I don't like the current trend and I hope they failed on streaming video game.
Problem is, smartphones still are shit for input (lol touchscreen) in any precise manners so what point is there in trying to run intensive games on it via streaming.
When OnLive was operational I had no trouble playing Batman (something something Arkham?) on their American servers from a MacBook Air in Sweden. I had a similar experience later with LiquidSky which was running regular games downloaded from Steam in their cloud and streaming video to my laptop.
If they do cooperate on creating a similar service, this will definitely be a net win for the gamers.
However, if that's what happens, and if the service is successful, we might end up seeing less games tailored to run on the console's hardware with low latencies, and we'll end up having cloud-optimised games (much like we have mobile-optimised games now). But this is pure speculation at this point.
Still I have a hard time imagining first person games, since I despise those with significant input lag. Even just on a local PC some games have a horrible delay unless you run very high frame-rates (quite a bunch of those Unreal Engine) and I always have a hard time getting into those.
Now with an additional 20-30ms network latency... for this to be close to acceptable, they would have to get the on server delay close to nothing at the very least. I doubt that will happen while games are still release on multiple platforms.
Your first point is key though. Piracy is a big deal. People can say until they're blue in the face that it "helps", or that "they wouldn't have bought the game anyway", and for some unknown percentage its true, and for some other unknown percentage its a straight lie.
If they go from "optionally play a game via streaming", to "this game MUST be played via streaming" eventually, they get the second group (the ones who WOULD pay if they couldn't get it for free) instantly.
Only if it doesn't suck, obviously.
People with sensitive hearing can tell the difference if a sound arrives <1ms too late and the sound system needs adjustment.
All my bets are on bubble on this one. Maybe it could be resurrected as a zombie to offer sandboxed streaming for all those crappy mobile apps.
And when you do, you will find out that there isn’t any actual issue. I’ve played games like The Witcher 3 via streaming over three years ago.
Input/output lag simply isn’t an issue, and I’m your average German VDSL user, nothing special.
From my point of view, there is no sense in any further discussion - the empirical data is already in.
It's like saying "Switching from a car to a bike just isn't an issue - I once rode all the way to my neighbour's house!"
Try playing a multiplayer first person shooter on the same service, you'll find latency is an issue.
By all accounts, the Doom demo that Google gave was plagued with lag.
The bigger limiting factor I think will be bandwidth, especially in America with the awful internet provider situation.
This would only be relevant if you had a direct fibre link in a straight line from your house to the service. It's packet switching\routing etc. that introduce latency. And even if you DID have direct fibre, the worst case scenario for distance - server on the other side of the world, but with no other latency accounted for - is about 200ms, even half of which would be noticeable in many games.
I'm not saying it's impossible to do game streaming, but you can't just divide the distance by the speed of light and say it's low. That's not really the issue.
Only when network is underutilized, and there is no throttling/bufferbloat.
In practice, it’s a lot worse than 200ms. 500-700ms are typical for servers on the other side of the planet.
Average Pings from Digital Ocean regions: BLR1: 251ms FRA1: 129ms SGP1: 206ms
Other sites: yahoo.co.jp: 179ms rakuten.co.jp: 138ms belarus.by: 167ms
As most things go, YMMV.
The problem with latency is that it adds up, lag from the server, lag from the TV, lag from my brain, lag to my fingers, lag from the controller to the machine, lag back to the server, lag in processing and then start again.
Client-Server lag might be acceptable in isolation but not as part of a system.
So not really anyone. And that's in an extremely latency friendly environment where the servers are just up the road.
I now play games with some friends and while I'm using a local PC, they use Shadow PCs (ie a VM with a GPU in a datacentre vaguely near them) and it's fine.
Maybe it'd be worse on an LTE/5G connection, but my phone is getting 25ms to the nearest Speedtest.net instance right now, which doesn't seem substantially terrible.
I would be much more concerned about the pricing of all of these things, than the technical issues. How many gaming subscription services am I going to need to add to my video subscription services?
From a hard theory network perspective, having higher bandwidth, lower latency, lower error rate on transmission path combined with closer geographical proximity to servers seem to help mitigate increases in latency.
However, if you argue the problem is solved by your empirical evidence gained by playing Witcher 3 of all games, I disagree with your point. Play a real time strategy game, a MOBA, or a first person shooter. 70ms~ (a report for Stadia reported 70ms-130ms, may be stale data) of lag is noticeable to people. On the other hand, I don't mind streaming a turn based game.
Certain games like rhythm games would also be pretty miserable with jitter...
There's the obvious DRM win. Accessibility will be higher with way lower or non-existent hardware costs. These probably look very enticing on a cooperate slide deck. Not to mention the subscription model.
On a more technical side game streaming is only really desirable for games that don't run well on commodity hardware (see AAA 3D action titles). Latency is very important for these titles and the general rule of thumb is that 50ms of input latency is readily noticeable to humans. Average home connections have ~20-30ms round trip latencies. These speeds are not a reality everywhere or consistent. Combined with hardware input, world simulation, and rendering latencies it can be difficult to get latencies low enough to be comfortable.
Modern games just have the client run its own world simulation and rectify with the server later to hide the network latency. This strategy won't be possible with streaming. Is there some alternate optimization that can be made for streaming? The architecture will definitely be simpler with the main-frame paradigm. Maybe if there's only one client it could be feasible to send some batteries included chunk of frames that can be easily constructed based on the next set of inputs. Is there any hint that progress can be made in this direction?
If the latency problem can't be solved, I'm definitely bearish or game streaming. Latency insensitive games for the most part are not difficult to run on commodity hardware.