Four year college graduates are selected from among a pool of people who did well in high school. Likewise, I suspect the employees you work with were not hired from among the ranks of the chronically unemployed. The US military is not a good counter-example. Members of the armed forces, both at the enlisted and officer levels, are better educated than the population as a whole.

This is not a matter of “disdaining” anyone’s “life circumstances.” It’s a matter of acknowledging that infrastructure work requires skills that not everyone has, just like programming does.