Let’s say it was done without any kind of special face recognition but was done purely by some funky nearest neighbor algorithm in high dimensional space. Would you still object?

The end result is the same, grouping things by commonality on different axes, one of them being faces. If the underlying algorithm didn’t even know what faces were, would it may a difference?

No. It does not "recognize" a face. If you take a few pictures of another Google user, it won't say - "Hey, here are all the pics of John". It will just allow you to lookup all pictures that match a face. Its clustering and not recognition.