You are tricked by what you see. You assume the other person is a person that looks like you but is rotated towards you, while in reality the other person would need to be a mirrored version of you. Your brain doesn't see that it's mirrored because you are used to see "yourself" this way.

I'm not looking at my reflection. I'm looking at you and your reflection(or a very good clone). Depending on my orientation, I might not even be able to figure out which figure is the mirrored image.