Perhaps, perhaps not -- but your examples do nothing to counteract the idea that it is, as every single one of the groups you've mentioned is, in the US, significantly advantaged over the broader, corresponding group that is identically described without the "white men" or "white male" limitation.

Now, in many case, the system is also stacked against the class described in the other part of the descriptor, such that the intersection of that class with the "white men" restriction is disadvantaged compared to "white men" more generally.

But that doesn't, in any way, change the fact that the system (in the US) is stacked in favor of white men.

> Why is it OK to assume the system is stacked in someone's favor?

There is a difference between "conclude" and "assume".

> How is that not prejudice?

If you assume in the absence of evidence, it is prejudice. If you conclude on the basis of evidence, it isn't prejudice (because there is no "pre".)