My pool cleaner, like most of us, has a finite lifespan. While it's certainly theoretically possible that he could have been a hedge fund trader for Goldman-Sachs, I think we can both agree that to accomplish that would've required a fair amount of luck somewhere along the way. He might, by grit and discipline and sacrifice, have managed to become a lawyer, and make more than he is making as a pool cleaner. But to become independently wealthy by age 30, he also would've had to be very, very lucky.
Looked at in this light, only a small percentage of the population has any chance at tremendous success.
Not only that, but only a small percentage of the population has ready access to the positions that offer the financial leverage necessary for that kind of success.
A few years ago I got on an excellent, high-paying contract. Had the contract lasted a few more years, I probably could have retired early. But neither the start of the contract, nor its end, were entirely (or even mostly) in my control. And while my work history was certainly a factor in getting the contract (though definitely not the only factor), it did nothing to make the contract last any longer.
Another important point is that one year ago I had US50k (but in brazilian reais) in debt because of a failed startup and a lot of difficult getting back to the work force. It was really hard to bootstrap myself out of that situation.
Let me play devil's advocate for a moment. Why do we believe this? I suggest that it's because we see examples of people who worked hard and got rich. But what if the truth is that most people work relatively hard---ie, keep busy at something. What if the pool of truly lazy people is actually rather small, which would make the pool of lazy, lucky people even smaller? Then it's entirely possible that success is all down to luck, but it would seem to us that most of the lucky people were also hard-working. But we would be wrong in associating the hard work with the success.
I firmly believe that hard work and applied intelligence is a key factor in making a decent living---doing a bit better than your neighbor, say. But getting rich, to the point where you're making 10x, 100x, 1000x the average, requires some extraordinary intervention somewhere along the line. I'm not so sure there is a strong correlation between the two phenomena.