Generally you can do this in system settings, but yeah, most people don't realize this a) follows the image no matter where it goes and b) is enabled by default

Honestly with some stuff we've seen lately, even reflections of notable locations in sunglasses or even eyes might be a concern. Not sure what you do about that.