It's not the the concerns are imaginary, it's that they are inconsequential. Everyone will have an opinion about everything they interact with; this is the human experience. You can't control the fact that others will form opinions about you and your actions, but you can control how you react regarding this.