Just to try to explain the question better, here's an example. Let's say I have 10 images and I want to find the most similar people among any pair of images. Do I need to run every pair of images (45 full comparisons) or can I pre-process the 10 images into something such that the 45 comparisons can be done in a less expensive way?
Generating 2 templates is many more times expensive than comparing those templates. However, as your dataset grows, generating templates grows at N, and the number of comparisons you need to do grows at N^2. So eventually, comparisons dominate.
In retrospect, it's not surprising that they're optimizing some parts with assembler - even SSE (Streaming SIMD (Single Instruction, Multiple Data) Extensions http://en.wikipedia.org/wiki/Streaming_SIMD_Extensions), which I hadn't heard of, but which is the kind of vector parallelism that gives supercomputers their speed.
1. Image to Image is a bit misleading because we are fairly strict in our online demo in pose requirements, so many faces are "non-suitable" because they aren't frontal.
2. It works much better for video settings (but that type of online demo is way too CPU intensive) because we can organize each person into continuous tracks and we only need them to become 'frontal' once to do matching. And the more they are frontal, the more data points we have.
Here's an example of recognition applied to video: http://www.youtube.com/watch?v=jsjf3IDXef8