Some people are just really bad at diligent hard work and lack the self control needed. The question is how to handle that without causing other issues.
A lot of the big cards are just dealt to you. Your parents, your skin, your religion, your school, your intelligence, your health, the neighborhood you grow up in--these things have a huge impact on what you do or don't have to overcome once you're old enough to take responsibility for your choices, and you don't get to pick them at the start. They have an outsized impact on what you even believe to be possible, never mind what you think is normal.
It's like how everyone says they would be anti-slavery if they grew up before the civil war despite the numbers just not supporting that. If I took away your advantages and made you do your life over you probably aren't going to break the mold and will end up roughly where everyone else of similar life circumstance ends up. What special spark could you even have? Because hard working and smart people are born in all walks of life but that only takes you to a local maximum.
You start having to consider that maybe white people are actually the superior race if you truly believe that it's not privileges because how else do you explain the vastly uneven distribution of success in the US?
Think it's more surprising how little of it is down to luck. Even working slightly harder reaps rewards quickly that outweigh the extra work.
It's most just most people are not even putting in the minimum.