I don't personally think "punching up is ok" is a valid viewpoint. But even if I did, I don't think white people are "up" or men are "up" in the way that the proponents of punching up consider rich people, or oppressors to be. Skin color isn't oppression. Gender isn't oppression. Oppression is oppression.
Economically, white people and men are clearly advantaged, and money is probably the purest form of power there is.
We can, but present day leftism is doing pretty much the opposite. You can't have race quotas without pushing race. Talking about "white priviledge" is enshrining the idea that your race defines who you are.
I agree we could decide to drop the idea of race. It would be really nice. Humans are always forming out-group hatreds, but it would be very nice if we could not consider another race the out-group. I don't think it's just as simple as just deciding to do away with the idea of race, any more than we could just decide to do away with the idea of money or like, the legitimacy of the court system. Sure, it's a social fiction, but it's self-reinforcing, which means it's hard to coordinate to get rid of.