News for hackers. Stuff that matters.
Re:The trauma of Facebook’s content moderators by msapaydin on Wednesday July 22, 2020 @03:00PM (#23920391)
What I am wondering, and this is probably a dumb question, is why this has not been automatized? Can't content moderation be done with modern and strong machine learning based systems? There must be plenty of training data on this, and just like a spam filter which does not require humans in most cases, this should also be automatable. Why is it not?
Re:The trauma of Facebook’s content moderators by nitwit005 on Wednesday July 22, 2020 @03:59PM (#23920930)
It has been. Most of these sites catch a ton of images and video automatically when they are similar enough to prior known content.

That doesn't matter from a staff point of view though. You have a queue to work through. You'll be putting in an 8 hour day dealing with the stuff the system doesn't catch. The automation just means they don't need as much staff.

Re:The trauma of Facebook’s content moderators by three_seagrass on Wednesday July 22, 2020 @04:16PM (#23921124)
Yep. One way to view automation with machine vision / perception is that it can cover ~85-95% of the true positives.

You're still going to get false positives and false negatives that need human review, and at a scale of Facebook, that's a lot of humans.

Re:The trauma of Facebook’s content moderators by msapaydin on Wednesday July 22, 2020 @04:53PM (#23921451)
I am just hoping that those currently uncovered cases will be "milder or more nuanced" cases that will be less damaging to the psyche of human moderators and will, once labeled correctly, improve the coverage rate of automated moderators.
Re:The trauma of Facebook’s content moderators by fsociety on Wednesday July 22, 2020 @03:36PM (#23920707)
The issue is that this is adversarial in nature. Certainly there are ML systems catching content before moderators.. however since it is adversarial people are pivoting to circumvent the automated systems.
Re:The trauma of Facebook’s content moderators by pixelbath on Wednesday July 22, 2020 @03:10PM (#23920483)
>this is probably a dumb question

Not dumb, but I do think it's not thought-through. You're proposing a simple solution to what is a hugely complex problem, and throwing ML at it just isn't going to work. To get "plenty of training data," a human still has to classify all of that, leading to the problem of viewing that much unpalatable content by a human. You also have to train your network, which requires humans to verify the accuracy of training, hence viewing the content again.

If it were as easy as text-based spam filtering, this wouldn't even be a discussion.

Re:The trauma of Facebook’s content moderators by msapaydin on Wednesday July 22, 2020 @03:12PM (#23920507)
I am guessing that by now plenty of data has already been generated which could be used to train future systems..
Re:The trauma of Facebook’s content moderators by pixelbath on Wednesday July 22, 2020 @03:28PM (#23920644)
I'm guessing you don't actually understand the complexity involved.

How does it differentiate between normal ranting and hate speech? How is hate speech classified in the US vs Saudi Arabia? How does it tell the difference between someone asking a child innocent questions and asking them sexually-related questions on camera? Does the algorithm get trained to flag videos about depression that might lead to suicide, or does it say they're supporting getting help FOR depression and leave the video?

What about subjects that aren't already in the corpus of "flag these naughty things"? You still have to get a human to look at those; most likely a data scientist who knows what the algorithm is doing and what needs to be done to correct the training. Machine learning as-is will not get us there, so in the meantime the only option is moderation by people described in the article. It can be outsourced elsewhere, but it's just shifting the responsibility to a different subset of people.

Re:The trauma of Facebook’s content moderators by msapaydin on Wednesday July 22, 2020 @03:37PM (#23920723)
Certainly there is more complexity than spam filters, and cultural differences, subtle nuances which are hard to train a system to auto-classify; things that need to be generalized to new cases that arises- that may be wrongly classified in the beginning, etc. Nevertheless such systems can be built from simple cases to more difficult ones step by step and gradually reduce the demand on humans to perform these tasks. Based on some other comments on this thread, I realize FB is already using these methods mostly in languages with plenty of data, and someone also in this thread has made an interesting comment about using machine translation to apply these systems to other languages. I think this will be less of a problem in the near future. (edited)
Re:The trauma of Facebook’s content moderators by SamuelAdams on Wednesday July 22, 2020 @04:21PM (#23921150)
It's a really hard problem to automate. You can scan photos against a list of hashes to sort out known bad / illegal content, but new content is generated every single day. Also YouTube has increased the amount of auto-moderation but that leads to many legitimate videos being taken down, with little recourse for those legitimate users. Some examples:

https://news.ycombinator.com/item?id=22487403

https://news.ycombinator.com/item?id=5068626

https://news.ycombinator.com/item?id=18742640

https://news.ycombinator.com/item?id=19065859

Re:The trauma of Facebook’s content moderators by ElliotH on Wednesday July 22, 2020 @03:22PM (#23920577)
What everyone else said, but also the presence of false positives means there'll be appeals. So you have a choice or ignoring the appeals and being accused of censoring people, or processing them - and that tends to need a person.
Re:The trauma of Facebook’s content moderators by racl101 on Wednesday July 22, 2020 @04:02PM (#23920978)
Not to mention the fact, that we'd be subjecting AIs to the worse that humanity has to offer. This is the kind of stuff that creates Ultrons or Skynets who decide that humanity is trash and that it ought to be wiped out. Lol.
Re:The trauma of Facebook’s content moderators by agentdrtran on Wednesday July 22, 2020 @04:00PM (#23920946)
It's not automatable at all, for example Facebook's guidelines change by the day and sometimes by the hour, and many cases are not able to be analyzed by machine as the error rate is too high.
Re:The trauma of Facebook’s content moderators by amelius on Wednesday July 22, 2020 @03:01PM (#23920400)
Perhaps because the training of these systems requires human supervisors?
Re:The trauma of Facebook’s content moderators by msapaydin on Wednesday July 22, 2020 @03:11PM (#23920490)
well, just as in spam filters, some supervisors would be necessary to automate things in the beginning, but eventually this should be less needed?
Re:The trauma of Facebook’s content moderators by amelius on Wednesday July 22, 2020 @03:14PM (#23920521)
Yes, one would hope so. My guess is that we're still in the data-collection phase. It's probably a tough problem because you want the number of false-negatives to be extremely small.