If race “doesn’t matter” to rich people, but is rather used as a tool by the rich to cause in fighting amongst the powerless (which is true and seems to be what you’re implying), then in fact race does matter as it affects how the powerless relates to each other. So racism matters, so you’ve contradicted yourself.