Look man, welcome to being an underrepresented minority / woman in the US. This is our every day everywhere we go. We always have to watch our tongues make sure what we're saying doesn't get us labelled as "uppity" or "thuggish" or "bossy" etc etc etc.
You're coming to terms with the fact that in any society you have to consider how other people will react to what you do. Congrats.
You can, however, get Damored though for expressing the wrongthink opinion.
If it's implausible for someone to imagine expressing their beliefs without being labelled a bigot then it's just as much a statement about the environment they're in as it is about the beliefs in question.
Again, the fact that the majority of people would call a given viewpoint bigoted is just as much a statement about the environment that calls the view bigoted as it is about the view itself. Plenty of mainstream liberal views would be considered bigoted in other developed democracies (e.g. a lot of European countries don't practice affirmative action). Conversely, plenty of mainstream conservative views would be considered bigoted by liberals (I'm going to go out on a limb and guess that I don't need to give an example).
[1] "Prejudice is an affective feeling towards a person or group member based solely on their group membership. The word is often used to refer to preconceived, usually unfavorable, feelings towards people or a person because of their sex, gender, beliefs, values, social class, age, disability, religion, sexuality, race/ethnicity, language, nationality, beauty, occupation, education, criminality, sport team affiliation or other personal characteristics. In this case, it refers to a positive or negative evaluation of another person based on their perceived group membership" https://en.wikipedia.org/wiki/Prejudice
Admittedly, I don't even like the term "conservative", like Naval would say, pinning labels onto yourself only forces you into static positions that you end up having to defend, even if you don't 100% buy into them.
I simply happen to have certain stances and ideas that are in opposition to the mainstream thought of SV. As much as I'd love to discuss them, potentially learning more about them myself and (gasp!) even changing my mind in the process, I'm not comfortable doing it. There's a high chance I will be immediately labeled as "racist / sexist / bigot / white supremacist / [fill in the blank]" for even contemplating disagreement on these topics (let's take "diversity" or how we're handling it as a society as an example) without any supporting evidence.
Unfortunately, as of today, bigoteering has 0% burden on the person making the accusation and 100% of the weight put on the accused. It's always safe to call someone a witch, but proving them wrong in one's defense is nigh impossible.
Ironically, the most open minded and considerate conversations about diversity that I've ever had were with black coworkers. I've learned plenty from them, changed my stance several times, realized I didn't know that much before.
The worst were almost always with overrepresented majorities who were "stepping in to speak up for their less-advantaged brethren". I've never learned anything from them on the topics they were so zealous about. It's the diversity version of "white knighting".
It's hard not to become jaded and assume that most people in this latter category don't actually care, and just want to establish themselves at the top of the moral hierarchy through vacuous virtue signaling. It's disappointing.