News for hackers. Stuff that matters.
Re:John Carmack: Latency Mitigation Strategies by modeless on Friday February 22, 2013 @01:35PM (#5265773)
I think head-mounted-display virtual reality is a huge missed opportunity for next-gen consoles. They could have really done it right in a way that PCs can't yet, with special hardware support for low latency and stereo rendering, and a guaranteed large audience so game studios could justify applying their full budgets to AAA VR titles. It would be a huge differentiator at a time when consoles seem to be converging to a very similar place. It could be as big as the Wii was.
Re:John Carmack: Latency Mitigation Strategies by ChuckMcM on Friday February 22, 2013 @02:31PM (#5266208)
I don't think they were a missed opportunity. In order for them to 'miss' they had to be 'possible' which they haven't been for a long time. Its only lately where various things have come together to get close. I have followed the 'head mounted display' technology closely since its early inception in the mid-90's as a means of providing military pilots with situational awareness.

The first problem with head mounted displays was resolution. Both in color space and in pixel space. Early displays were monochrome (either green or red based on LED display drivers) and had a roughly 256 x 256 dot pitch. It was ok for targeting reticules and basic instrumentation display (like an artificial horizon). Early work also wanted transparency (look through) displays because there wasn't a way to display what you were looking at in enough fidelity to do both the visual field and the indicator field.

Later a company called Colorado Micro Displays (CMD) came to market and started offering color, but 320 x 200 was the best resolution they could do. Higher resolution displays cost tens of thousands of dollars and were essentially hand crafted out of unobtainum. Direct retinal illumination displays attempted to get past strict display resolution issues (a friend of mine worked on one of the x-y positioning device for doing this, it had 500 angstrom repeatability! And was, as expected, insanely expensive)

It really hasn't been until recently when DLP type systems at sufficient resolution to eliminate the x-y type beam director have come to market. Combined with better LED phosphors to bring the cost of an RGB display with better than VGA (640 x 480) resolution into something practical.

The incessant push to higher resolution phone screens has created the capability to make opaque but high resolution screens which can enable something like the Oculus Rift type displays. That requirement stems from how far away from the eyes you have to put the screen and thus the torque moment that is applied to your head. There was a great system at NASA which used 10" displays but the 'head mounted' part was more like sticking your head into some weird steadycam kind of device.

Finally there is the challenge of both high fidelity and high frequency head orientation technologies. Prior to about 2005 the best you could do was a laser gyro for motion and an accelerometer for inertial reference. Doing that at the necessary frequency (typically 1000 updates per second) didn't become cost effective until about 2009.

Now however, many of the technologies have finally matured to the point where OR can be done in small quantities for perhaps $2K/unit.

A 'game console' is struggles if it costs more then $300. So as a 'controller' option, and even as the 'whole console' option, these sorts of glasses are still about 3 - 5 years from hitting a price point that makes them the 'killer' peripheral. And of course because they aren't here yet, they can't really have 'missed' :-)

Re:John Carmack: Latency Mitigation Strategies by modeless on Friday February 22, 2013 @03:34PM (#5266602)
$2K/unit? That's a gigantic overestimate. High quality head tracking is practically free today thanks to advances in cameras and MEMS sensors; you don't need laser gyros. The Oculus Rift costs $300, and is almost good enough for a home console. The only major improvement needed is a better display, which needn't cost more if sourced in quantities large enough, which Sony or Microsoft could definitely do. $500 for a complete system with display, controller, and console is definitely achievable, and that's the price point the PS3 started at. VR is absolutely feasible today.

As I said further downthread, I think a lot of people got disillusioned with VR because there's a lot of crappy hardware out there. Even the expensive stuff is crap. I tried Canon's augmented reality system at SIGGRAPH last year and the latency and FOV were awful, despite the $120,000 cost. But it doesn't have to be that way, and the Oculus Rift is the proof.

Nintendo saw the possibility of doing something new with the Wii, using then-new technology (MEMS sensors, tiny low-power cameras) to make an old, lame, expensive concept (motion control) work for the mass market, and was rewarded handsomely for it. If someone had that foresight with VR they could be making a killing right now.

Re:John Carmack: Latency Mitigation Strategies by ChuckMcM on Friday February 22, 2013 @03:50PM (#5266697)
I love your optimism.

I too look forward to the OR system shipping. Go through the latest parts list for the OR device and check on availability 2 years ago. Things are moving along and that is great, they weren't there when Nintendo was using MEMS accelerometers and CMOS camera modules in their remote.

Re:John Carmack: Latency Mitigation Strategies by micheljansen on Friday February 22, 2013 @01:43PM (#5265824)
I think the monumental failure of Nintendo's Virtual Boy pretty strongly illustrated that if you can't do this perfectly, you shouldn't do it at all. Within the current constraints (including budget), making a "good enough" product is clearly not feasible. As this article points out, even a few milliseconds too much causes massive problems (worst case: severe nausea – car sickness).

On top of that, it's not even clear that people want this. Full-on Virtual Reality is a bit antisocial and hyper realism isn't the only thing that makes games immersive and/or fun.

Re:John Carmack: Latency Mitigation Strategies by modeless on Friday February 22, 2013 @01:51PM (#5265886)
Virtual Boy pretty strongly illustrated that if you can't do this perfectly, you shouldn't do it at all.

Absolutely. But the Virtual Boy was clearly before its time. The hardware just wasn't ready.

Within the current constraints (including budget), making a "good enough" product is clearly not feasible.

I think that's clearly not true. The response from everyone who's tried the Oculus Rift has been that real VR has finally arrived. And the Oculus Rift is one guy's prototype; the resources of a company like Microsoft or Sony could drastically improve it in several dimensions. Specifically, they could source a higher-res, higher frame rate, lighter, lower-latency OLED display, and connect it to rendering hardware with all the latency-reducing features John Carmack just described. These features aren't necessarily expensive from a hardware POV; it's just that nobody's seen the need to implement them yet.

Re:John Carmack: Latency Mitigation Strategies by neverm0re on Friday February 22, 2013 @02:04PM (#5265985)
> Specifically, they could source a higher-res, higher frame rate, lighter, lower-latency OLED display

None of those are actually remotely as important as the fact you're still using your head as a dumb camera joystick, there's no ability to track eye movement and the mismatch between what you're seeing and your inner-ear pressure is going to make a number of people throw up.[1]

These are fundamental problems that have been a part of HMDs for the last twenty years of me playing with them and the Rift addresses absolutely none of them. The Rift itself is not significantly different than its HMD contemporaries, it's actually aiming for the low-end gamer market and so far seems to be mostly pushing 'What's old is new again' without having made significant progress along the way. The most interesting part of the Rift kit is the tracker, not the actual HMD itself.

[1]: In a study conducted by U.S. Army Research Institute for the Behavioral and Social Sciences in a report published May 1995 titled "Technical Report 1027 - Simulator Sickness in Virtual Environments", out of 742 pilot exposures from 11 military flight simulators, "approximately half of the pilots (334) reported post-effects of some kind: 250 (34%) reported that symptoms dissipated in less than 1 hour, 44 (6%) reported that symptoms lasted longer than 4 hours, and 28 (4%) reported that symptoms lasted longer than 6 hours. There were also 4 (1%) reported cases of spontaneously occurring flashbacks."

Read up on 'Simulation Sickness' for more details.

Re:John Carmack: Latency Mitigation Strategies by modeless on Friday February 22, 2013 @02:18PM (#5266112)
Eye tracking would be awesome but it's not required for good VR. I fail to see how "using your head as a dumb camera joystick" is a problem; that's the whole point of VR.

Motion sickness is a real problem for some people (but not all). With low latency and thoughtful game design I think it can be mitigated. The bigger problem is the social acceptability of blocking your entire field of vision for long periods of time. I won't pretend that VR doesn't have problems, but the payoff is large enough that the problems are worth tackling.

Re:John Carmack: Latency Mitigation Strategies by neverm0re on Friday February 22, 2013 @02:34PM (#5266223)
> I fail to see how "using your head as a dumb camera joystick" is a problem; that's the whole point of VR.

Perhaps you should try playing some VR games for a while. I clocked quite a few hours playing the old Virtuality SU2000 games like Dactyl Nightmare and I've idly kept up with HMDs. The chief problem is that without eye tracking, it's incredibly UNFUN to use your head for movements that your eyes could otherwise have done for you. Mind you when the HMDs were much heavier back then it sucked a lot more, but it's still pretty shitty not being able to glance aside. Nope, gotta move your entire head for absolutely everything related to what you're currently seeing, if you move your head for any reason you can't maintain focus on objects naturally, etc.

> Motion sickness is a real problem for some people (but not all).

Probably not something you should underestimate. See Nintendo's 3DS launch and about-face on their stance on pushing 3D once they found a small but significant percentage of their users could not actually see the 3D effect. This lead to policy that the 3D effect could not be used for anything related to actual gameplay mechanics, reducing it entirely to an optional gimmick.

Simulation sickness affects even more people than the 3D issue. It's a real problem if you want to go mainstream.

Just to clarify my position, I'm not against the Rift nor do I have anything against HMDs. I simply see the Rift as a step along the way to whatever device truly popularizes the tech. I don't think we're there yet.

Re:John Carmack: Latency Mitigation Strategies by modeless on Friday February 22, 2013 @03:28PM (#5266569)
it's incredibly UNFUN to use your head for movements that your eyes could otherwise have done for you.

That's only a problem if your FOV is too small. With a large enough FOV and a light display there's no reason why eye and head movements shouldn't work exactly as they do in real life. The Oculus Rift has twice the angular field of view of the SU2000 in both dimensions, for 4x the subtended angular area.

Edit: Oh I see, your complaint is about using the head tracking to control a game, e.g. by pointing a gun. Yes, I think that's a bad idea. Head tracking should only control the camera. All game interaction should happen through a controller. A motion controller like the Razer Hydra would probably work well for this.

if you move your head for any reason you can't maintain focus on objects

Again, only true for crappy hardware. With a good enough display, 120 FPS, and low latency, there's no reason why tracking moving objects shouldn't work just fine.

I think a lot of people got disillusioned with VR because there's a lot of crappy hardware out there. Even the expensive stuff is crap. I tried Canon's augmented reality system at SIGGRAPH last year and the latency and FOV were awful, despite the $120,000 cost. But it doesn't have to be that way, and the Oculus Rift is the proof.

Re:John Carmack: Latency Mitigation Strategies by neverm0re on Friday February 22, 2013 @03:44PM (#5266655)
We seem to be so close to agreement here that I'll just have to vote you up and take you at your word regarding the Rift. It's entirely possible my experiences have all been unfortunate ones. I've not tried the Rift yet, though by all means I will. :)
Re:John Carmack: Latency Mitigation Strategies by modeless on Friday February 22, 2013 @04:19PM (#5266908)
Well, my word about the Oculus Rift is secondhand, since I haven't received mine yet. And I haven't tried the Sensics device you mentioned below; that hardware looks quite nice, so it would be quite disappointing if that level of device still wasn't good enough for a great VR experience. My opinion may change after using the Rift for a while, but given what people are saying about it I'm still optimistic :)
Re:John Carmack: Latency Mitigation Strategies by Randomoneh on Wednesday February 27, 2013 @10:34AM (#5292826)
Modeless, can you please answer this "trick" question? If a flat piece of cardboard that is 10 inches wide occupies 10 degrees of my horizontal FOV, how many degrees of my horizontal FOV will 20 inches wide flat piece of cardboard occupy? Both cardboards are positioned at the same distance from my eyes, of course.

Thanks.

Re:John Carmack: Latency Mitigation Strategies by drivers99 on Friday February 22, 2013 @03:08PM (#5266442)
> The chief problem is that without eye tracking, it's incredibly UNFUN to use your head for movements that your eyes could otherwise have done for you. Mind you when the HMDs were much heavier back then it sucked a lot more, but it's still pretty shitty not being able to glance aside.

Isn't that more of a problem of field of view? With a wide enough field of view, you could just look at whatever you wanted to. Basically make it like real life, where what you look at is the combination of where your head is pointed, which the computer needs to use to update the screen, and where your eye is looking, which the computer doesn't need to care about. Or are you thinking of using eye tracking to something else, like determining what you're pointing at like a mouse? Yet another option you have in VR is what you're targeting, like how Dactyl Nightmare uses the gun you're holding to map directly into the virtual world's gun. Using the head position to AIM rather than just LOOK seems like a bad way of doing things.

Re:John Carmack: Latency Mitigation Strategies by neverm0re on Friday February 22, 2013 @03:36PM (#5266609)
Yes, you're absolutely right that a lot of what I'm complaining about would be solved with sufficient FOV and things certainly have improved since the SU2000 system in that regard. But we're not at 'sufficient' at the moment, at least in my experience based upon trying more recent examples, like an obscenely expensive Sensics kit. It's possible that it will be sufficient long before eye-tracking winds up in consumer HMDs.

But yes, the other part of my previous remark regarding 'dumb joystick' was more directly related to using the head for aiming/pointer duties, which also unfortunately has cropped up before and I quickly conflated the two issues together.

Re:John Carmack: Latency Mitigation Strategies by TruthElixirX on Saturday February 23, 2013 @04:38AM (#5269133)
>This lead to policy that the 3D effect could not be used for anything related to actual gameplay mechanics, reducing it entirely to an optional gimmick.

Citation? I had not heard this about the 3DS.

Re:John Carmack: Latency Mitigation Strategies by marshray on Friday February 22, 2013 @02:10PM (#5266040)
> mismatch between what you're seeing and your inner-ear pressure

How does your inner-ear pressure change such that it cannot be fixed with better display and rendering?

Re:John Carmack: Latency Mitigation Strategies by neverm0re on Friday February 22, 2013 @02:20PM (#5266122)
Because the working theory on what causes simulator sickness is related to the inner-ear detecting motion. If your eyes are detecting motion, but it does not line up with what your inner-ear pressure is telling your body, the end result is the body emptying its contents under the theory that it's been poisoned. Simply improving the quality of the picture isn't going to solve that.
Re:John Carmack: Latency Mitigation Strategies by ealloc on Friday February 22, 2013 @02:57PM (#5266367)
It seems to me the oculus rift can accrately account for all head motion when you are standing still. There is no need for eye-tracking: All the rift has to do is detect all head bobbing & rotation, and update the view acocrdingly. Your eyes can still look around on the lcd screen.

What the rift _can't_ do is account for long term linear acceleration (eg in a car or walking around), it's true. If you had a large empty room to walk around in with the rift on, even this could be accounted for, however.

Re:John Carmack: Latency Mitigation Strategies by untog on Friday February 22, 2013 @01:52PM (#5265896)
I think VR is fundamentally broken, because you're going to try to react to things in a way that doesn't work. With a head-mounted display, the game can react as you move your head. Great. But then you'll instinctively try to walk towards something and it won't work (or you'll hit the wall). Same with picking things up, etc.

It seems like it's deep in the uncanny valley of experience.

Re:John Carmack: Latency Mitigation Strategies by purplelobster on Friday February 22, 2013 @02:33PM (#5266215)
It's not difficult to learn not to start walking when you're in VR. I've seen some people lean left and right when they play an FPS for the first few times, but they get used to it. VR with a simple controller though is just not good enough. You'll need arm/hand tracking and probably a gun controller for shooters. If you can track 6 degrees of freedom of the gun controller and 6 DOF of the head, then you have your first immersive VR right there.
Re:John Carmack: Latency Mitigation Strategies by WesleyJohnson on Friday February 22, 2013 @02:20PM (#5266131)
I think it's just a matter of time before the other pieces to the VR puzzle catch up. Omni-directional treadmills are already in existence and are a basic step towards allowing you to walk in a VR environment. Developments in prosthetics with 'feeling' could help pave way towards shoes, gloves or other garments that could provide you with the sensory experiences of walking on different surfaces or picking up in-game items.
Re:John Carmack: Latency Mitigation Strategies by untog on Friday February 22, 2013 @02:26PM (#5266173)
But at that point you've gone from a games console being something that takes up a small amount of space underneath your TV to something that takes up an entire room. From requiring you to pick up a controller to putting on an entire outfit.

I'm just not sure the average consumer is in any way ready/willing for that.

Re:John Carmack: Latency Mitigation Strategies by purplelobster on Friday February 22, 2013 @02:35PM (#5266228)
I think the first successful VR you'll see will be closer to an arcade machine than a home console.
Re:John Carmack: Latency Mitigation Strategies by untog on Friday February 22, 2013 @02:44PM (#5266272)
Agreed. But that's at odds with the OPs assertion that " head-mounted-display virtual reality is a huge missed opportunity for next-gen consoles."
Re:John Carmack: Latency Mitigation Strategies by stcredzero on Friday February 22, 2013 @03:21PM (#5266525)
> I think the first successful VR you'll see will be closer to an arcade machine than a home console.

I was just thinking about this. My development rig has 2 27" screens. If I added a 3rd one, this would cover most of my field of view horizontally. What if someone sold a console rig packaged with 3 screen and a mount? (Work with IKEA to produce a compatible cheap desk.) This kind of thing is already done and is awesome for vehicular games. (Racing, tank, aircraft...) A 3 screen rig also could still strike a kid as "awesome," so would draw money from the wallets of parents.

Or, for arcade-type settings, how about a seat with a projected 360 dome, HOTAS controls, and motion simulation?

Re:John Carmack: Latency Mitigation Strategies by BystanderX on Friday February 22, 2013 @02:17PM (#5266101)
I agree, it'll certainly be uncanny.

Except, we adapt. I think Uncanny Valley will slowly shrink - Things will improve, and we'll get used to some of the oddities.

I mean, he mentions it in the article: People were used to 400ms latency and still had fun. Is there a much tighter threshold here? Sure. But I'm all for it, and I'm sure a lot of people are. Not everyone, but things improve and change.

Re:John Carmack: Latency Mitigation Strategies by mtgx on Friday February 22, 2013 @02:09PM (#5266024)
Well Oculus Rift should support everything from PC's to consoles. But I agree. Stuff like Oculus Rift is the true next-gen computing. The PS4 and next Xbox leave me pretty cold, especially when I already have a laptop that can already play all the latest games.
Re:John Carmack: Latency Mitigation Strategies by nitrogen on Friday February 22, 2013 @02:18PM (#5266114)
I think the PS4's insane memory bandwidth (8GB of memory on a 170GB/sec GDDR5 bus) makes it a very interesting contender against PCs (dual-channel DDR3 maxes out at 34GB/s).
Re:John Carmack: Latency Mitigation Strategies by roc on Friday February 22, 2013 @02:29PM (#5266194)
Given the PS4 is using an off-the-shelf x86 processor, how long do you think it'll be until other OEMs are pitching laptops with these same CPUs that have GDDR5 controllers for main memory?

How long until Intel matches? How long until basically every CPU supports it and every manufacturer has one or more such systems?

Re:John Carmack: Latency Mitigation Strategies by nitrogen on Friday February 22, 2013 @03:08PM (#5266449)
They may be using an off-the-shelf x86 core, but their actual CPU die may have all kinds of customizations on it (like the addition of a GDDR5 memory controller). AIUI GDDR5 memory is much more expensive than DDR3. So it may happen eventually, but I don't think it will be any time soon, and believe me, I really want PCs to be better than consoles in every way.
Re:John Carmack: Latency Mitigation Strategies by roc on Friday February 22, 2013 @03:54PM (#5266723)
It just seems to me that whatever they're doing with AMD, AMD is going to be able to farm out to other parties. Given that both MS and Sony are using AMD cores paired with GDDR5 memory controllers, it seems almost certain that there's nothing approaching an exclusivity contract to prevent AMD from now selling that bundle to any maker of gaming PCs or SteamBox builds or whatever else.

And the current/historical cost differential between DDR and GDDR5 certainly seemed to be a considerable problem back when these specs were first rumored. But the following things suggest that's not likely to be an issue preventing it from hitting other gaming PCs: 1. Sony managed to spec 8GB of it into a box they simply can't afford to subsidize greatly at the outset. [1] 2. Sony and MS both need to know the cost of any expensive components will drop massively over the next couple years. 3. PC gamers readily and regularly accept higher prices for their machines than console makers could ever stomach.

So I don't see it becoming common. Truly very few non-gamers would care about the performance delta. But it seems like a thing that will be on offer from multiple vendors, for feasible prices, before long. (Particularly given that AMD could really use any edge against Intel.)

[1] People wrote off a lot of the rumors on that basis alone. If Sony is confident that it can not only afford GDDR5 for system memory, but that it can afford so much, there has to be a massive price cut incoming.

Re:John Carmack: Latency Mitigation Strategies by Tuna-Fish on Saturday February 23, 2013 @12:02AM (#5268633)
> Given that both MS and Sony are using AMD cores paired with GDDR5 memory controllers

They are not. MS is using DDR3 at 256 bits * 2100MHz for 68GB/s bandwidth.

Re:John Carmack: Latency Mitigation Strategies by Tuna-Fish on Saturday February 23, 2013 @12:00AM (#5268630)
The memory bandwith is perfectly normal for GPUs, though. And the CPU will mostly suffer from the GDDR5, because it has much higher latencies.