Add this to the difficulties of finding people that good in the first place (even more so said people willing to work for a large company and moderate at scale), and you've got a bit of a problematic situation:
- If you try to hire 'quality' moderators, then you likely spend millions of dollars a year on moderation, without any real guarantee that the moderation will actually be good (since different communities and groups disagree on it, cultural differences come into play, etc) - If you try to outsource the moderation to some low cost region, then you get poor quality moderation due to cultural differences and a lack of understanding of the subject matter and community. - And if you try to automate it, you arguably get an even worse situation, since false positives and negatives are everywhere and often there's no way to appeal the situation when they get things wrong.
Then things get even worse when you realise that different communities interact and have different standards...
Moderation is one problem of gray area, there is never a clear line between what is acceptable content and what isn't. Not only is the community and their opinion on what's acceptable changing, the moderation rules are a moving target and at scale you inevitably have government pressure to moderate certain content that the government doesn't approve of.
From every attempt I've seen tried over the years in tech, moderation is a losing battle and the main reason building large platforms for user-generated content is such a nightmare.
Mastodon tried this, specifically with federation as an attempt to decentralize. As long as people try to use it similar to a centralized platform like Twitterz the moderation problems only get worse. You end up with more moderators with less accountability all trying to moderate both their smaller circle and everything from the outside world too.
The only way I see fixing it with a different network model is by going for a fragmented system. Think 90s era forums, very group is its own small bubble and is never attempting to he part of some unified global town square.