Modern facial recognition uses landmarks, much like humans do. First it identifies a face, which is essentially a flat 2-d plane. It's able to identify locations of the nose, ears, mouth, chin, etc. It then looks for relationships between these features (e.g. how wide is the mouth, how far apart are the eyes, etc).

It's a much simpler problem than the general image detection. Imagine how many different types of dogs you can encounter at various angles, sizes, colors, perspectives. For that it's much more of a black box as your model would have to support detecting a dog from behind, side, front, above, sitting down, walking, jumping, standing, dogs with long tails, short tails, long hair, short hair, etc. Now think about a face. Most faces are very much alike.

https://github.com/ageitgey/face_recognition