"all the people I know who inherited their money"

Selection bias. You're specifically selecting for people whose only money is money they inherited; failures, in other words.

There's a lot of children of rich people who are quite successful in their own right. Bill Gates, for example, did not come from a poor home. And the other thing you have to watch out for is reversion to the mean. It's unlikely Gates' children will outdo him.