I've been accused by a friend for not doing enough in a much less egregious situation (but also in the category of morality and equality), and understand the social difficulty in broaching such a topic, especially if you don't know them well.
It's easy to say "do the right thing" on an internet forum to a stranger, especially if you yourself haven't been in this social situation where you know what the right thing is in a vacuum, yet our survival instincts push us towards not stirring the water. I think your reaction thus far is completely rational given such an alien situation (which is where I found myself in the past).
I just want to say that your internal turmoil is being discounted too much by other commenters. I hope that you can find internal peace and understanding of where you stand and what you believe in for yourself, not because someone tells you that's what people "should do", before doing or saying anything, because that is the only way we can sustain both our moral and emotional health.
However, part of your response gave me pause: "I just want to say that your internal turmoil is being discounted too much by other commenters."
I would offer that we cannot underestimate the internal turmoil, stress and emotional trauma of the aforementioned "housekeeper" (slave), who sleeps on a mat in the kitchen, and that any internal turmoil by those of us with freedom pales in comparison.
Immediate, unambiguous action is the only moral response to slavery.
"I need to keep rooms spare for when friends visit."
"When does that ever happen though? Come on man, they would understand if they had to stay nearby or squeeze into a room. [Housekeeper] is there every day. She is a gem and works so hard. I couldn't do that to someone. I think if you have a better supported employee or helper or whatever, they're more likely to be happier around your family and feel naturally interested in their role."
"She doesn't mind."
"I bet she would, but do you really think she'd feel comfortable raising that with you? You should do the right thing. Think about how valuable a comfortable room is to you. Everyone should be able to appreciate that and not sleep on a mat."
It seems there's an opportunity to meet my friend in person soon. I'll use it to ask about what exactly happened. It's not the kind of thing that comes across well electronically, you really need to be there in real time looking at the reaction. It also needs to be managed properly because this friend is in a somewhat fragile state for completely separate reasons. The last time we met this other issue meant nothing else could even be discussed.
But the key takeaway for my kids is his parents aren't perfect. Hopefully he will have learned that in other ways by the time he's old enough to understand this situation.