> On top of that, who the fuck would ever use a product if their private notepad was going to be read by a human moderator? There is no easy answer here.
Besides, we've gotten to a point where it's not clear if A. every single thing being ingested as data to create a profile of me off (current situation, e.g. Gmail) is better than B. Only a portion that has been flagged being reviewed by a human.
There's a lot of low-hanging fruit which humans-in-the-loop would be used for to give far better results. A very obvious example being Meta's key role in the Rohingya genocide. Plenty of outsiders banged on Facebook's door, but they wouldn't listen, costing thousands of people their lives. There's lots of such "obvious" cases. Content moderation is extremely difficult when it comes to things that toe the line, sure, but I wasn't really talking about those.
> And, at a large scale, false positive rates and false negative rates are usually in a trade-off, even when people supposedly agree on what is supposed to happen. (Which they also don't actually agree on.) That's especially true when companies are using software and statistics to help them make judgments, or when they're not spending billions of dollars to have decisions made by domain experts, at leisure, with rights of appeal.
There is a course of action that leads to far better results; human review. But that cuts into the $billions of quarterly profits, so we don't want that.
The idea that it's unaffordable to these companies is a complete fabrication. It should be forced upon them. If that halves their profit, so be it, it just means their margins have been too high due to offloading negative externalities unto society.