They are using openface: https://cmusatyalab.github.io/openface/#openface

The library gets the picture as input and spits out a 128-element vector. The more similar the vectors are for two pictures the more likely they are of the same person.

You don't need to train it, there is a pretrained version available.

This reminded me of a news article I read about "eigenfaces" as a facial recognition method way back; here's its corresponding Wikipedia page: https://en.wikipedia.org/wiki/Eigenface

Really, there would be "eigenshapes" for any arbitrary configuration of 3D space. That we can smoothly transition from one face to the next in a higher dimensional matrix is no surprise.

My guess is that they use siamese cnns or autoencoders and look at the latent space. Also, just straight up training a deep neural network on 20 million classes isn't impossible. We've developed an in house technique that greatly accelerates this. Keep your eyes peeled on arxiv for a publication soon ;)

What authors should we be looking for?

Also, the essence of it is sampling the "labels" (the people's face ID) similar to Word2Vec through similarity. When applied to people's face ID, the key phrase is "In feature space, faces of a feather flock together".

Then, use a recurrent ConvNet at the end to pump out probabilities for each class (in this case face) with a low/tractable number of parameters.

Spent a couple of hours of interesting reading. It sounds like faces cluster in high dimensional space. From the perspective of a curious spectator I would love to see a blog post or article that answered:

What would pairs of maximally distant faces look like?

What would the mediods of a large dataset that was clustered into two clusters look like? Would it be a high level feature that we'd recognize like male or female?

Would faces that were mediods of the cluster be considered more attractive?

Would it be possible to build a similar app that allowed you to identify an individual from memory? Cluster the dataset and present the user with images representative of the mediods of the clusters. Discard images from clusters that weren't selected and re-cluster. Repeat until a handful of images remain.

Could that technique work as a virtual sketch artist and identify a person's doppelganger?

Deep, Derp, et al.

:)

Tapabrata Ghosh

They could just be using a feature space for face representations that a model is trained to generate given images, which would reduce the identification problem to a n-dimensional clustering problem.

I'm not as familiar with the models in face recognition but something analogous to word2vec may produce reasonable results.

Not much info in the video but they did train for 6 month to get their first good results. There also seem to be neuron combinations for race and eyes...that's pretty much it.