>Is the review not human? Do they run it through the same algorithm a second time? Do they lie about the review?

Yes, yes, and yes. There's a context-illiterate AI deciding what you can say or not, even if the later heavily depends on context.

And just like it'll trigger a bazillion of false positives, it'll also trigger a bazillion of false negatives; someone can easily spread hate through Facebook, unimpeded, by simply encoding language in a way that the algorithm doesn't understand, but humans do; or with simple irony.

I think the algorithms are just minimum standard efforts which provide enough plausible deniability for FB to be able to argue that they provide safeguards on bad content.

Pretty much.

People don't often realise, but users are a resource just like any other. And if a resource is abundant, you'll often sacrifice a bit of that for the sake of something else.

So... sure, bad content (false negatives) and being handled unfairly (false positives) sours users, prompting them to leave the platform. And both things are bound to happen, if you put an algorithm reviewing stuff instead of a person. But since there are so many users, Facebook is better off doing this crap than actually spending money by hiring enough people to actually review the reports, checking the context to decide if they apply or not.

And you'll see other businesses doing exactly the same decision; that's why, for example, Reddit became such a shithole, and people outright complain about the Anti-Evil Operations all the time. Twitter is likely the same, dunno. And it shows that we cannot and we should not rely on mass social media, we should be sticking to smaller alternatives.[/soapbox]

The human review is probably someone who doesn't understand the language verifying that a word is present for fractional cents on Mechanical Turk or something.

It could well be that they are paid very little, the majority of reports are true positives, have a quota to make, and expect no consequence for false positives. If its outsourced they may have a bad grasp of the language as well.

> no consequence for false positives

This could be the key point. Penalties for letting something bad slip past, but no penalties for falsely flagging. It’s a pragmatic solution, but it is almost by definition inhuman.