Banks would send black children to the homes of white people with pamphlets saying things like "you know who is moving in," warning about an imminent decline in property values due to black people moving in. Banks would come in after the pamphlets with agents who would low ball these white-owned houses, then flip them for profit by charging black people huge prices for the very same homes. It was entirely rooted in racism, no way about it.

Imagine not wanting to have someone in your neighborhood only because that person is black, without you needing to know anything else. That sole fact of their skin color makes up your mind that you don't want them in your neighborhood, without asking if they're a good person, an honest person, a friendly person.. they're simply disqualified without you asking any additional question.

You can imagine if someone doesn't want black people in their neighborhood how they might feel about their children marrying one.

I don't know how to read people's minds, so I tend to gravitate towards a functional definition of racism which measures it based on someone's action - i.e. if someone being treated in an unfair manner based on their skin color.

I don't see anything wrong with that. If I move out to a place without blacks, because it's a place without blacks and I am the one that moved, why should I have to accept them following me? That's not racism. That's wanting to be left alone. It's also not unfair. Forcing someone to move is unfair. Moving out on your own and preventing people you don't like from moving in isn't unfair.

There's also a game-theoretic stalemate that you have to break with active policy change.

To the typical mid-century white homeowner, "a black family moved into the neighborhood" wasn't the beginning and end of it. There were only two equilibria: either you live in a segregated white neighborhood or you live in a predominantly black neighborhood. (I openly admit there's nothing wrong with living in a predominantly black neighborhood, but this was probably not the most common opinion among mid-century white homeowners.)

Why? Fundamentally because there was a structural shortage of good housing for black people. If most neighborhoods don't allow black residents, having a neighborhood that does allow them causes a surge of demand. Even if you're not racist enough to move out just because a single black family moved into your neighborhood, some of your neighbors are, and many of their homes will be bought by black families, which will trigger your slightly-less racist neighbors to move out, and so forth. This was deliberately encouraged by "blockbusters", who would buy homes from white families at a discount and then price-gouge the black families who ended up moving in for a massive profit. But blockbusting only works if you have the pre-existing shortage of unsegregated housing in the first place.

You end up in a feedback loop:

(a) Moderately racist white homeowners want to keep their neighborhoods segregated because otherwise they'll live in black neighborhoods. This causes:

(b) Most neighborhoods are segregated white neighborhoods, causing a housing shortage for black families. This causes:

(c) Overwhelming surges in black demand to move into any individual neighborhood that desegregates. This causes (a).

The "solution" was to immediately, globally, across-the-board, outlaw housing discrimination so the pent up housing demand can disperse. I put "solution" in quotes because most American metro areas are still extremely segregated. They might be statistically segregated by 90/10 ratios instead of the 100/0 ratios that existed before, but people have to go significantly out of their way to completely reverse it, unless there's some other economic incentive. For example, gentrification has "desegregated" lots of majority-nonwhite urban neighborhoods through the sheer force of white hipsters wanting to move into them.

“Racism” is an overloaded term. The idea you’re describing (racial separatism) is definitely in the general category of racism.

In fact, most racism gets expressed as racial separatism. Segregation was literally racial separatism.

One problem is that this literally turns into a question of ethnic territorialism, and can easily be extended from neighborhoods to entire states (Oregon was founded as a whites-only state) to entire countries (eg the notion of sending freed slaves back to Africa, which is how Liberia was founded).

It’s also just not that hard to express even the worst racism in mere separatist terms. In fact, that’s where most racism starts.