"I would say several of those examples actually do increase increase risk to others, second hand smoke, being incapacitated by drugs or alcohol can often put others at risk for a number of reasons, etc."

which is why all those things are limited by law. I can't smoke in public spaces, I can't drive drunk, etc.

They are not 'rights' as you seem to argue, and they are limited/regulated just like a million other behaviors. Wearing a mask is no different. Personally, I'd say mask regulations are the equivalent of seatbelt laws - maybe uncomfortable, or inconvenient but basically harmless.