I don't disagree, but "the poor people who clean them" are doing that because it pays better than they could get elsewhere, otherwise they wouldn't do it (assuming no coercion on the billionaire's part, which would be illegal).

If the billionaires magically ceased to exist and poor people owned the apartments, I don't think they'd be paying the other poor people as much to come clean them.