You have to think about the effects of what you are saying.

Let's say you have the "non-racist" slave owner. No matter what they personally believe about the equality of individual humans, they create a world where the people who come after will believe fundamentally different things about races.

You can't say, slavery IS wrong, but we are going to do it anyway because we are selfish. The act is racist, because the people who come after you will justify your decisions and cement a racial worldview that you contributed to.

Racism is a set of beliefs. Tornados, door knobs, planets, these things can't have beliefs - only people can - which is why the argument makes no sense.

I really hope you read this CNN article in full but here is a quote that I believe directly applies to your argument:

> "When some whites talk about racism, they think it's only personal -- what one person says or does to another. But many minorities and people who study race say racism can be impersonal, calculating, devoid of malice..."

http://www.cnn.com/2014/11/26/us/ferguson-racism-or-racial-b...