>If you're born into the middle class you'll have a much easier time getting further in life than if you're born into a poor, crime-ridden neighborhood with bad schools.

That's a caricature of the problem. You don't need to live in a bad neighborhood. You only need to live in a community where people don't talk about starting businesses, don't know how to start businesses, or never made friends with anybody who owned a successful business. No connections = low chance of success.