Statistically, there are measurable and not really small differences in attractiveness of parties to different genders, races and religions.

The implied association likely have a lot to do with those visible differences.

Free throught does not mean that everyone finds same appeal of same policies and groups regardless of their differing interests and experiences. Quite the opposite.