We merely want to make a proof of concept that inspires users with a mind-opening sense that the world's data can have higher dimensionality than just 2 dimensions.
I (Jon) read the novel Flatland when I was a kid, and have ever since been yearning to see in more than just 2 dimensions. This project is an attempt to open users eyes in the same way.
--
With the goal defined as proving a concept, it becomes apparent that we do not need to permanently solve every challenge with AR in a scalable way.
We merely need to kludge around them (for example using tons of computing power) enough to make a convincing temporary experience:
1. Patterns of marks or lights can be placed on the outside of the headset, detectable by cameras in the environment.
2. True. Luckily we can achieve a kludgy demo by having one entire rendering computer (with graphics card) per depth slice, and selecting between their outputs as a dumb switch, post-rendering.
3. The plenoptic camera from lytro.com should provide the necessary post-capture-focusable images. We are meeting with Prof. Marc Levoy today to ask about partnering.
--
We merely hope that once the concept of voxel vision has been proven, that all future AR systems will feature it as a standard mode of interaction.
1. In order to be able to render virual objects on top of the real image (camera image or directly to the retina) you should know the pose of the eye with respect to the outer world. For example if you are a surgeon and want to overlay an MRI scan on the patient, you need to know the pose of the eye (camera) in respect to the patient and this is a VERY hard task as everybody doing computer vision can tell you. You only mention how to find the pose of the eye with respect to the head but this is not enough.
2. Processing large 3D models like 3D scans requires a lot of computational power. It won't be very portable.
3. You will also need a depth image along the camera image in order to achieve some of your goals (like bluring the image according to the focus distance). Do you have any idea how difficult this task is?
You have cool ideas, but should be more realistic about your goals. Also reading some things about augmented reality and computer vision will help you a lot to understand how difficult this is.