As, well, a human being, yes: I am just fine with refraining from volunteering my time or attention to people who fit or do not fit some criteria. It's called "freedom of association."