Stereotypes. They exist for a reason, it's just that we think they are true more often than they are, so we end up making the wrong choice much of the time.

Most elite Northerner whites are just as insulated as they think Southerners are, when it comes to traveling within their own country. In fact they typically know jack shit about Southern culture. There was a time when Southern accents were considered deliberately slow, enunciated, cultured. Yes, at that time whites were at the top of the class food chain, and yes most of that white privilege was racist or at least depended on it. But the accent is orthogonal to racism. We've just come to associate one with the other, using a stereotype.

Another stereotype is white Northerners will pretty much assume white Southerners, identified as such by their drawl, may be a racist.

Assumptions are traps.