Isn't another difficulty found because different sets of populations exhibit different characteristics?

Young-Old, Tall-Short, Poor-Wealthy, Educated-Unschooled, ColdClimate-HotClimate, etc?

All your examples are characteristics, not preferences. The model gets the data 'there aren't many women in STEM (end of pipeline)' so it doesn't allocate resources to girls who are interested at the start of the pipeline. The prediction -> action cycle generates a self fulfilling prediction that winds up at a bad local minimum (maybe the model needed better leakyREU functions).

This is the same problem why your netflix recommendations just show you either the stuff everyone else is watching or the stuff super similar to the stuff you've already liked, it's hard to make predictions that are both novel and useful. There has to be room for exploring new domain space.