It's even worse, really. If you call somebody "white" trash, you're noting that this is a white person in a situation that you would expect of a nonwhite person. Expecting nonwhites to be poorer in the US is rational. Expecting nonwhites to be badly-behaved, stupid, and criminal is just racist. I'm sure you'll object to this, but I guarantee you don't call anybody "black trash."
While I still think "redneck" carries a negative connotation in many places, it also seems to be a lot less negative nowadays, And I'm more likely to hear it in the context of self identification or (while somewhat still derogatory) to refer to a kinda idealized version of an unruly and unsophisticated, but fun rural people, and to many, the new redneck stereotype is a sort of role model. Hell, half the "rednecks" I knew in HS were middle class kids who's parents had good job and lived in nicer suburbs than my parents could afford.
There have been some artists trying to take back the "white trash" moniker and use it in a "yeah that's what I am, what about it?" sort of way.