Nobody particularly cares about people doing high-contagion activities like going to a gym, for the people in the gym themselves, the ethical problem is that they're very few degrees of separation away from people who can't afford to be exposed. Even if you drive by and mind your own business, that gym almost certainly has a blast radius of unnecessary infections and deaths.