You sound like a cool, mindful person.

I think a strong part of what you're describing is that you need to be on point with your non-verbal communication; you need to both tell people that you won't take offense to constructive criticism, and signal it with a lot of cues.

Telling people "this is an example of prejudice, this is how I react to it, if you don't act like this you're clear" is a pretty smooth move too.