'Members of the team quickly showed signs of stress — anxiety, drinking, trouble sleeping — and eventually managers brought in a therapist. As moderators described the images they saw each day, the therapist fell silent. The therapist, Mora-Blanco says, was "quite literally scared."'
And:
'Beth Medina, who runs a program called SHIFT (Supporting Heroes in Mental Health Foundational Training), which has provided resilience training to Internet Crimes Against Children teams since 2009, details the severe health costs of sustained exposure to toxic images: isolation, relational difficulties, burnout, depression, substance abuse, and anxiety. "There are inherent difficulties doing this kind of work," Chen said, "because the material is so traumatic."'
butters was in charge of filtering cartman's comments so cartman would only see material that would be unable to offend him
cartman seemed to flourish with his new perceived support and popularity and butters deteriorated
i thought it compelling to look at how they chose to do butters
'this material is offensive! you should deal with it instead of me'
cartman was characteristically unconcerned about butter's well being as long as the supportive comments kept coming in
i understand the satire loses some weight when you acknowledge that those that hire these moderators are mostly companies with a vested interest in keeping their users blissfully unaware
10,000 individual users being inadvertently exposed to 1 piece of upsetting material could potentially mean 1 moderator then has to review 10,000 pieces of upsetting material
also, the deluge
mailroom syndrome, it just keeps coming
honestly, computer vision is the only way to do this sort of moderation with any semblance of success and unfortunately i feel all of this human debris is just considered collateral damage until we get it functioning
I think they'll go down as two of the greatest commentators of our generation.
i failed to because i refuse to link to that hulu run abomination
bah, south park, the show where the creators themselves put their work on the internet to be viewed freely and for free
what a unique expression of the creators' personal beliefs
a stance that had a profound impact on me, and my peers
when hulu bought the rights i felt my skin grow cold, but i held out reservations in the hope that someone at hulu was just a fan and wanted to improve south park studio's backend whilst respecting the implicit wishes of the creators that their work is free and readily consumable by anyone regardless of manufactured scarcity or paywalls
now this shit is the new normal:
"Safe Space"
This episode is currently not available at South Park Studios
and There was an error playing this protected content. (Error code: 3365)
i lament the loss of a future's exposure to the challenging ideas innate to the showhulu has zero respect for the series or its creators opinions on how open it has always been, and should be
fuck you hulu
it would be unsurprising that a subset of moderators develop PTSD-like symptoms related to that experience.
Is that really the case? Most moderation is done on textual-based forums, like the one we're on right now. I'm having a hard time imagining the right combination of words on a screen to generate trauma in the requisite quantities for someone to wind up with a clinical disorder because of it.
Personally, I've been on both sides of this coin. I've been the guy kicking trolls off of a decent sized board, I've been the guy getting kicked off because I annoyed the wrong person.
At no point did it ever progress beyond internet drama. The meanest, nastiest person I could imagine could type in words along the lines of how they'd like to kill and fornicate with my mother, and my, and most other people's response I wager, would be "That's cool, kid. Bye now. Ban"
Maybe there's a case to be made for becoming jaded after a while of dealing with the worst (hang out on the meta Stack Exchange sites for a publicly visible example) - but PTSD symptoms? To me, that both overly glorifies the troll "i can type words so good that I can make other people have legitimate mental breakdowns!" and makes light of the suffering by people who actually have PTSD (who have seen things like people dying). It feels like a lack of perspective, brought on by spending a lot of time online. Unlike real life, disconnecting from online drama is always a button press away.
The article clearly describes moderators watching videos flagged as offensive, and these videos included "amateur and profession pornography" and many that contained "child abuse, beatings, and animal cruelty". Furthermore the article discusses moderators having to deal with videos shot during the Iranian revolution in 2009, including the murder of a young woman, "a shaky cell-phone video captured her horrific last moments: in it, blood pours from her eyes, pooling beneath her."
The article presents many more examples of the enormously disturbing tasks assigned to moderators. Based on tens of years in clinical practice I'd consider it likely that viewing such material, especially seeing it frequently, can precipitate acute and chronic stress disorders in vulnerable individuals. I understand the point this is only hypothetical in the absence of systematically collected data, call it a clinician's hunch if you want, but still I'd put money on it.
Rereading the article confirms for me that it is an excellent piece of journalism, perhaps even exceptional in the current era. Moderators were exposed to far more than words, in a way similar to 911 operators suffering exposure to trauma without being at the scene. Forum moderation has very little in common with the subject of the article.
You are severely underestimating the effects of this kind of experience on people in such positions. You should learn more about stress disorders, the complexity of which is bound up with the unique and highly variable attributes of individuals. Some people are much more resilient than others making it misleading to generalize about how people will respond to given situations.
Most of all, I strong suggest you (re)read the story and if capable, allow yourself to empathize with the plight of the moderators.
Uh, why? Am I a counselor?
> allow yourself to empathize with the plight of the moderators
To what end? What possible constructive purpose is served by allowing myself to share (or wallow) in the negative emotions described by others? Does it stop "bad things" from happening? Does it lessen their pain? Does it lessen anyone's pain? Or does it increase the overall amount of suffering in the world?
brb, going to voluntarily take a job that makes me feel bad, so I can cry about it on the internet...
Even on a textual-based forum like reddit there are two subreddits which I usually visit (/r/syriancivilwar and /r/combatfootage) where one can see links to videos of dead people or worse (that's because very nasty things happen during wars). I could very well imagine someone getting PTSD-like symptoms after having moderated videos/links to videos of children who have just died because of chemical-weapons attacks. That is what nightmares are made of.
For starters, this is not true. Stuff that happens online has real world consequences. People get hired, fired, meet people they marry, get doxed and have their personal contact info put out there and on and on. The Internet is not disconnected from real life to the degree that your comment makes it sound.
I was a military wife for two decades. I lived in the High Desert where I dealt routinely with coyotes and large spiders and poisonous snakes, etc. I have spent time on health forums where deaths were so routinely reported that they had their own practices for how to deal with death, their own culture surrounding that so to speak. These experiences have left me somewhat thick skinned.
I read accident claims for over five years at an insurance company. Because of the aforementioned background, it was rare for me to be disturbed by the detailed medical and police reports that I read for up to 8 hours a day, five days a week. Many of my coworkers found the work far more stressful than I did.
But I would absolutely not want to have to moderate the photo and video stuff described in this article. And I would absolutely feel personally threatened by people making ugly threats of the sort that you dismiss with "That's cool, kid. Bye now. *Ban".
I'm pretty thick skinned, but I'm also a woman. Most of the people who are moderators are women. A lot of this nastiness is absolutely aimed at women. I think it hits a lot closer to home for most women than it does for most men. Women tend to routinely deal with genuine threats to their welfare, such as stalking, harassment and rape, that men tend to be largely exempt from.
When you recognize that your work impacts real people and how safe they are, that also can be a burden. When I read accident claims, in most cases, no lives hung in the balance. That is part of the reason it did not bother me to read them: I was not deciding whether or not to report anything to the police or otherwise take an action that impacted someone's personal safety. I was only deciding whether or not to cut someone a check and whether or not to refer the claim to the fraud department.
In a few cases, I was trying to get it paid rapidly because the customer was broke and facing having their power cut off and nothing to eat in the house. But, really, what I did mostly did not have a sense of "And if I do the wrong thing, someone could wind up killed, raped, fired from their job etc." What moderators do can have such impacts. That kind of responsibility can wear on a person. Perhaps not you, but plenty of people would find it wearing.
The Internet is something of a Pandora's box, it seems.
I once started watching a movie with Monica Bellucci. Some way into the film, she is followed into a tunnel and brutally raped. I couldn't make it through the scene without getting sick to my stomach and feeling the need to throw up. It even made me want to cry. It was far, far too intense. I had to turn it off.
I can't help but feel awful there are people whose jobs are screening such things recorded from real-life events, who don't get to just turn it off. There's always another clip in the queue.
While, the tone could be construed as lacking in empathy, and the article clearly states that users are exposed to videos, the question can definitely be rephrased as a valid one.
When reading the accounts of the moderators, I of course immediately felt sorry for them - it must be pretty harrowing to watch a constant barrage of child porn, murders and gore (think being on /b/).I imagine it would leave you feeling pretty jaded and losing faith in humanity.
But my second thought went to all my friends who are doctors and see terrible injuries and ailments on a daily basis, or criminal lawyers who review cases which revolve around the gutter of human behaviour. What is the PTSD incidence rate amongst these two professions?
What sparks PTSD? Is it the context by which the triggering events are presented? Are lawyers and doctors more resilient to this as there is an expectation to be exposed to these types of things, and therefore the professions are self-selecting? Is it the relentless barrage of content which affected the moderators? Or was it the lack of empowerment wrt doctors/lawyers to make an impact on these things?
Going forward what can we do to better prepare moderators for this?
An effective automated approaches will provide a big advantage in narrowing down the "judgement space" that must decided by humans. To the extent that's possible, the key benefit provided is reducing the exposure of moderators to stressful situations as the article describes, and indeed that's a very helpful development.
As the article points out the whole domain of moderation practices is a minefield. But now I wonder if there isn't also a risk of automated review making classification errors re: user behavior. Plausibly automated systems can be tuned finely enough to avoid serious errors, and support human oversight to catch and more easily resolve edge cases. Automated moderation systems will need to have such qualities in order to be able to reduce human burden as intended.