I think there is a strong punishment system for the non-labelers. People that feel strongly about this stuff are really rabid, and seem to spend all their time calling people out on perceived slights.