I genuinely don't really care about things like this. I mean sure It'd be a little odd if my neighbour would stare at me but I generally don't mind, as you said no harm done