If you believe that, you should look more into the history of ethnic discrimination in the US. "Holding people back" is a systemic issue, and one that is still present.

But it is already bad enough that Afroamericans had been kept from acquiring wealth by various means, judicial and extrajudicial for much of the last century. They are closing the wealth gap, but extremely slowly.

Ignoring that history is part of a basically racist system.