Head turning without thinking about it would probably induce motion sickness. I think something the earlier Rifts taught us (through low framerates, lack of head tracking, etc.) is that our brains freak out if there are any significant inconsistencies between our movements and what our eyes see.
I know tech. has some way to go before it gets there, but that is the missing piece that would make this the next level of computing (and probably one that would cause Half Life 3 to finally show up).