News for hackers. Stuff that matters.
Re:A Framework for Moderation by snikeris on Wednesday August 07, 2019 @10:02AM (#20634740)
> Stand on the street corner all you like, at least your terrible ideas will be limited by the physical world. The Internet, though, with its inherent ability to broadcast and congregate globally, is a fundamentally more dangerous medium.

The notion behind this seems to be:

"We must protect the idiots of the world from terrible ideas."

Why does this rub me the wrong way so much? I guess it's such a depressing view of our fellow humans?

Re:A Framework for Moderation by commandlinefan on Wednesday August 07, 2019 @12:25PM (#20636196)
> Why does this rub me the wrong way

I’m old enough to remember when Judas Priest (the 80’s rock band) was accused of hiding subliminal messages in their songs convincing their listeners to commit suicide, after a kid did so while happening to be listening to Judas Priest. It seems absolutely ridiculous in hindsight, but people believed it, and it was taken completely seriously at the time: https://www.rollingstone.com/music/music-features/judas-prie.... The logic here is essentially the same: some internet message boards are “egging somebody on” to do something that they would not otherwise have done, which results in death, so the internet message boards themselves are to blame, and must be censored.

Re:A Framework for Moderation by fredley on Wednesday August 07, 2019 @12:57PM (#20636539)
Unlike the supposed subliminal messages, the egging on is literally on display, in plain English text.
Re:A Framework for Moderation by commandlinefan on Wednesday August 07, 2019 @02:54PM (#20637710)
Not really, though. If somebody says "immigration is a danger to our country", that could just as easily mean "vote republican" as it could mean "shoot some random people".
Re:A Framework for Moderation by atoav on Wednesday August 07, 2019 @04:00PM (#20638303)
This is based on the assumption that message boards are a one-to-many media like books, radio, movies, computer games, when in fact they are a many-to-many media, which under all empirical observation displays inherently different dynamics than traditional passive consumer media.

There is a reason we dedicate academic branches to these phenomena — they are not nearly as straight forward as argued in this thread.

Re:A Framework for Moderation by the_gastropod on Wednesday August 07, 2019 @01:09PM (#20636671)
Religious texts, advertisements, and tall tales have been "egging people on" to do things they otherwise would not for thousands of years. Judas Priest and Internet message boards are not new in this respect.

There is something unique, I think, about these message boards, though. I think most people would agree, the overwhelming majority of humans are reasonably decent—they're not going to murder others, are not overtly racist, etc. And throughout human history, these truly 1-in-a-million horrible people have been isolated, unable to find fellow horribles to befriend and egg each other on. The internet has changed that. No matter how unusual or awful your interests may be, it's trivial to find another group of people who share in these thoughts and interests. I don't know what the solution is, or if there even is one, but this does feel like something truly new.

Re:A Framework for Moderation by mindslight on Wednesday August 07, 2019 @01:24PM (#20636824)
This "egging on" has been going on throughout the centuries, and has mainly been getting more centralized. Nowadays most people seem reasonably decent, but when exposed to a repeated message of "social consensus" will still generally support going to war against a foreign country.

The novelty is that violence is now being mobilized against subcultures within larger society, rather than comfortably against foreign cultures in far off lands. What we're witnessing is the democratization of media-incited violence, from the general democratizing aspect of social media.

Previously, the privilege was reserved for large corporations that paid the traditional media big bucks to support their massive wars. Now individuals can invest sweat equity in a microcult to accomplish the same on a small scale. And the reason we keep hearing so much about it is that the incumbent gatekeepers are not happy about the competition!

Re:A Framework for Moderation by Psyonic on Thursday August 08, 2019 @01:11PM (#20646355)
I found this really insightful. The parallels are definitely there.
Re:A Framework for Moderation by laughinghan on Wednesday August 07, 2019 @01:51PM (#20637076)
I'm confused by your stance. Is it your position that ISIS recruitment videos don't work? They spend a huge amount of effort on online recruitment, but 100% of the people that go join them would all have voluntarily done so anyway?
Re:A Framework for Moderation by bdhess on Wednesday August 07, 2019 @10:14AM (#20634840)
It’s not implying that many or most people are idiots or untrustworthy. But that there are extreme outliers at the fringes.

Before the internet, they’d mostly be unable to find each other. With the internet, they can build self-reinforcing “communities” on the basis of hate, intolerance, or really anything. If they just stayed in that bubble it’d be fine, but unfortunately sometimes they bring machine guns to department stores.

Re:A Framework for Moderation by stcredzero on Wednesday August 07, 2019 @11:18AM (#20635478)
With the internet, they can build self-reinforcing “communities” on the basis of hate, intolerance, or really anything.

As you say, such "communities" existed before the Internet. It's just that now, they can better organize to affect the real world. I think Forrest Gump's observation applies here. Even 8chan had some sort of ideology justifying what they said and did. Even the worst scoundrels of history used words like "love" and justice." (And yes, Godwin's Law comes into play if we go down that rabbit hole! So let's not.) If we are to gauge how toxic a group is, I think we need to go to first principles and measure how driven by viral outrage an online group is.

It's natural for people to come together because they are outraged. However, when continuing the cycle of outrage, driving more attention, driving group membership, driving more outrage continues for its own sake, there is a problem. It shouldn't be outrage and attention driving a group. It should be higher principles.

Unfortunately, outrage is a fundamental fuel of virality in 2019. Today, we need to be skeptical of actors who lean too heavily on the cycle of outrage, whether they're openly reprehensible, or if they merely dress their movement in high language.

https://www.youtube.com/watch?v=rE3j_RHkqJc

In 2019, the world has long known to be skeptical of religions gone wrong, which we call cults. We need to develop the same kind of skepticism to groups actually ruled by outrage, not principle.

Re:A Framework for Moderation by mrec on Wednesday August 07, 2019 @10:29AM (#20634989)
Counterpoint: BoingBoing recently reported on French politicians trying to get "stigmatizing agricultural activities" covered by their proposed hate speech laws. As ever, if it can be abused, it will be abused.

https://boingboing.net/2019/07/10/hateful-content.html

Re:A Framework for Moderation by nickloewen on Wednesday August 07, 2019 @11:36AM (#20635675)
Thanks for the reference. But as far as I can tell, this didn’t pass. And I disagree with your conclusion—showing that people may seek to abuse anti hate speech legislation isn’t sufficient to prove, I don’t think, that the legislation will be abused.

Edit: removed comments on US “ag gag” laws cited in the linked article, after I read a little more and learned that they may not be so similar as was implied in this piece.

Re:A Framework for Moderation by frereubu on Wednesday August 07, 2019 @12:12PM (#20636025)
This kind of context is incredible important. Boris Johnson spent a decade in Brussels writing up as many stupid suggestions from fringe EU politicians as he could, all of which failed if they got anywhere near a vote (which most didn't), and managed to create the impression that most EU laws were like this. It even got the point where in the recent Conservative leadership election he told a bare-faced lie about EU regulation (to do with smoked fish of all things) and got away with it. Just because someone suggests something doesn't mean 99% of their colleagues don't just roll their eyes and say "anyway, as we were saying..."
Re:A Framework for Moderation by mindslight on Wednesday August 07, 2019 @01:03PM (#20636600)
... eventually Boris Johnson gave up and went home, never to be heard from again.

Neither viewpoint absolutely is true, but "As ever, if it can be abused, it will be abused" is the better heuristic. The people pushing the abusive laws are doing so as their day jobs, while their victims necessarily have other livelihoods that constrain their efforts. They don't simply propose the topic once, see that it has no support, and go home. Whomever they're working for has longstanding goals, and each win they get is a step normalizing a bit more of their desired regime.

This is why even when the Internet community strongly rebukes new Internet censorship laws (SOPA being a recent example), the same bullshit just comes back again - maybe tweaked to be a little less aggressive, maybe just using a better outrage-event as justifications. And every time it passes, it's a tooth in the ratchet - heads they win, tails we play again. Capitulating a round just accelerates the process.

Re:A Framework for Moderation by commandlinefan on Wednesday August 07, 2019 @12:08PM (#20635983)
I’m seeing this sort of implicit dialogue in all of this that goes like: “How much moderation/censorship would have been required to have prevented the El Paso shooter? However much that is, is how much we need”. This line of reasoning ought to scare the hell out of people, since we don’t actually have the slightest clue what actually set him off (he was on 8chan?) Even if I could bring myself to believe the underlying premise: that, if not for 8chan, this guy would have gone on to be an ordinary, law-abiding, productive member of society, but that 8chan itself brainwashed him into becoming a mass murderer, I would _still_ be opposed to shutting it down (and then shutting down the next thing that replaces it, and the next thing that replaces that…) because we’re losing everything else that goes along with it.
Re:A Framework for Moderation by frereubu on Wednesday August 07, 2019 @12:18PM (#20636097)
I sort of see where you're coming from, but surely there are a plethora of other places with light moderation and far less hate speech where people could go to discuss things? You seem to be making a slippery slope argument, but I don't really see the start of the slope. I'd also be interested to know what you think of this quote from the Cloudflare CEO:

'If this were a normal circumstance we would say "Yes, it’s really horrendous content, but we’re not in a position to decide what content is bad or not." But in this case, we saw repeated consistent harm where you had three mass shootings that were directly inspired by and gave credit to this platform. You saw the platform not act on any of that and in fact promote it internally.'

Re:A Framework for Moderation by grawprog on Wednesday August 07, 2019 @02:30PM (#20637466)
I find the way 8chan's being grouped together as a cohesive whole in these discussions odd. 8chan couldn't be more chaotic and disparate in the kinds of people that were there. There were many different people with a huge range of beliefs. Most people that hung out there from what i could tell were just lonely and had made it their special club outside society. I never went on /pol/, but outside /pol/ plenty of people couldn't stand them. They'd spam other boards with their nonsense and i seen plenty of anons tell them to straight fuck off repeatedly. On the boards i used to use, there were far more threads that had nothing to do with politics or race than ones that were.
Re:A Framework for Moderation by commandlinefan on Wednesday August 07, 2019 @02:33PM (#20637494)
Yeah, exactly - "there are some bad people on 8chan, therefore it's 8chan's fault" is really fallacious reasoning, but it's pretty popular right now.
Re:A Framework for Moderation by frereubu on Thursday August 08, 2019 @06:36AM (#20643268)
I think the issue there is that 8chan didn't seem to do anything about the nastier elements of /pol/. As Prince said, they only took that decision because they felt 8chan were ignoring the problem, which I think is a real one. I'm conflicted about this issue, as any sane person should be, but on balance it felt like the right decision.
Re:A Framework for Moderation by buildzr on Wednesday August 07, 2019 @12:20PM (#20636135)
> You seem to be making a slippery slope argument, but I don't really see the start of the slope.

Okay, so you ban 8chan. Now they move to another chan... there's about 5 different "bunkers" running the same or similar software to what powered 8chan and with many of the same users that various boards have moved to right now. Some of the larger of those bunker chans are actually on distributed networks.

So now what? You have to ban distributed networks? You shrug your shoulders and say "oh well, we tried"? This is where the slope comes in. There's no good ending to it. Worse, as we've seen with 8chan, the farther you push people from the mainstream, the more extreme they seem to become.

Re:A Framework for Moderation by frereubu on Thursday August 08, 2019 @06:40AM (#20643288)
The argument in the linked article, which I found compelling, is that by doing this you significantly raise the bar for participation. That cuts out the casual audience, reduces the audience size for these grandiose manifestos (which seems key to them being posted there), and sends a signal that this hateful content isn't generally acceptable. Of course there will always be truly horrible people but, on balance, I think the benefits outweigh the costs. As I said in another comment, I'm conflicted about this, as everyone should be, and will keep reading about the pros and cons. But at the moment I'm convinced that this was the right move.
Re:A Framework for Moderation by buildzr on Thursday August 08, 2019 @08:46AM (#20643976)
> you significantly raise the bar for participation.

This simply isn't true, it's trivial to set up clearnet proxies into such services and they can be easily treated as disposable.

The message you send them seems more to be that they need to hunt and hide more for "the truth". The need for that further contributes to extremeified views. It turns out pushing people into deeper and darker recesses doesn't improve their sociability. You can see this difference from communities that used to exist for the content that spawned these on reddit, Twitter to 4chan leading to the spawning of Voat, Gab, 8chan and others. The farther down the line you go the more extreme you get.

Re:A Framework for Moderation by pmart123 on Wednesday August 07, 2019 @12:21PM (#20636141)
In some capacity, is it better to let something like 8chan exist given law enforcement and the government can actively monitor it and try and prevent such events from occurring? Would changing the laws around what constitutes domestic terrorism allow law enforcement to act more swiftly to premeditated descriptions of such acts on similar forums? Even in the NZ shootings, would it be better to let something like that be uploaded and broadcasted as anyone thinking about going a religious service sees the eminent threat of a terrorist attack in real-time? I don't have answers to these questions given the large trade offs, but I do believe it is another side of this debate to discuss.
Re:A Framework for Moderation by frereubu on Thursday August 08, 2019 @06:44AM (#20643302)
I see what you're saying, and if these were small little online groups of, say, ten people, perhaps I'd agree. But one issue here is the audience size it gives people. Media guidelines (although they rarely follow them) say that you shouldn't publicise the names or images of people who commit these crimes, because that's a large part of what they want. The mainstream media are complicit in this too, and they should shoulder a good deal of the blame for that. Online forums are an easy target for the mainstream media because it means they don't have to examine their own part in the problem, but that doesn't mean the place where these events are initially publicised and encouraged shouldn't be tackled too.
Re:A Framework for Moderation by swebs on Wednesday August 07, 2019 @11:09AM (#20635393)
There are so many counterpoints to that statement that it's ridiculous. The American revolution, the Russian revolution, the Ides of March, every single coup throughout human history, every single cult, religion, or terrorist organization throughout human history. The only thing the internet changed is the fact that now the NSA has the entire log of all their conversations.
Re:A Framework for Moderation by pmart123 on Wednesday August 07, 2019 @12:23PM (#20636164)
This is an important consideration that often gets missed in these discussions in my opinion. The methods and means to organize a group of people against oppression often are the same ones to rally a group of people around hatred and violence.
Re:A Framework for Moderation by stcredzero on Wednesday August 07, 2019 @01:07PM (#20636655)
As I note above, the entire difference, is whether or not the group gets past the cycle of outrage amplification and onto higher principles, or whether it merely pays lip service to principles, and it's actually outrage that's the "tail wagging the dog."

It's entirely analogous to the religion <--> cult spectrum. The former is found beneficial by many and has much to do with community. The latter is really about exploitation and control. In 2019, we need to be skeptical of groups that use outrage to spread, much as we're skeptical of cults. (And this, despite how flowery or "good" their language sounds.)

Re:A Framework for Moderation by disgruntledphd2 on Wednesday August 07, 2019 @10:15AM (#20634857)
But surely this is the price we pay for the connecting abilities of the Internet? I'm not sure we can start censoring and still retain all the benefits.

Regardless of how much I hate 8chan, I'm still pretty worried about the precendent these kinds of actions set.

Re:A Framework for Moderation by snikeris on Wednesday August 07, 2019 @10:19AM (#20634879)
> Before the internet, they’d mostly be unable to find each other. With the internet, they can build self-reinforcing “communities” on the basis of hate, intolerance, or really anything. If they just stayed in that bubble it’d be fine, but unfortunately sometimes they bring machine guns to department stores.

I don't think the internet is fundamentally different here. You could have made the same argument when the printing press came out.

Re:A Framework for Moderation by JoeAltmaier on Wednesday August 07, 2019 @10:34AM (#20635028)
When a problem gets orders of magnitude bigger, it can be considered a fundamentally different thing. Requiring different solutions.
Re:A Framework for Moderation by the_snooze on Wednesday August 07, 2019 @02:21PM (#20637375)
I feel this is something that technically-inclined people tend to downplay too easily. Scale matters. Scaling something up changes its very nature and implications. Just because we can enjoy party poppers doesn't mean we're cool with flashbangs exploding near us.
Re:A Framework for Moderation by islon on Wednesday August 07, 2019 @10:29AM (#20634986)
The printing press is heavily moderated.
Re:A Framework for Moderation by mrkstu on Wednesday August 07, 2019 @12:50PM (#20636454)
Where and how exactly is traditional printing regulated? I can buy a high end typesetting laser printer and print anything I want on it- if it isn't child porn/top secret/libelous I can print anything I want that I've originated.

Can I steal other's content and sell it, no, but I would call existing regulations almost non-existent for original content.

Re:A Framework for Moderation by stcredzero on Wednesday August 07, 2019 @01:02PM (#20636586)
Where and how exactly is traditional printing regulated?

Publishers. There are documented cases of writers and genres being blacklisted and suppressed. (Though in fairness, you might find more than a few of them unsavory.)

I can buy a high end typesetting laser printer and print anything I want on it- if it isn't child porn/top secret/libelous I can print anything I want that I've originated.

Sure, but good luck with the promotion and distribution.

Re:A Framework for Moderation by mrkstu on Wednesday August 07, 2019 @01:58PM (#20637157)
Neither of which are regulation/moderated- just expensive or needing domain expertise and manpower.

It may be 'moderated' in the sense of an optical cable vs a copper one- by its nature there is impedance in the copper one limiting its potential carrying capacity. In that sense the internet vs traditional publishing have a different ability to scale. But Hitler's rise didn't need the internet- Mein Kampf found fertile soil due to economic depression and already extant racist attitudes latent in the populace.

China finds ideas contained in books by Locke/Milton/Paine or the Magna Carta to be extremely dangerous- and that they are and I'm sure they're be ingested wholesale in Hong Kong right now, where the Great Firewall isn't in full effect.

We need to counter evil thought with the only available agent that will actually work- better argued thought that seizes on vigorous ideas that instill a deep abiding respect for their fellow men/women and the potential of civil society to overcome the divisions we've placed among ourselves.

Re:A Framework for Moderation by laughinghan on Wednesday August 07, 2019 @02:06PM (#20637242)
Moderated. Not regulated.

You could spread your message further by printing lots of posters and putting them up in your city, but it would obviously not be comparable in reach to the Internet. Newspapers that do have reach comparable to the Internet all choose to exercise their legal right to decide for themselves what is too hateful to spread. Cloudflare has now chosen to exercise that same right, to a tiny extent.

Re:A Framework for Moderation by buildzr on Wednesday August 07, 2019 @10:30AM (#20634994)
Considering now that many 8chan users have moved to ZeroNet and Tor based sites, what would shuttering such things mean? I worry that it sets an extremely dangerous precedent, I don't want a Chinese-style great firewall, blocking all peer to peer and anonymization services just to keep out some channers, the vast majority of whom are relatively harmless.

I think attacking this from the access to guns angle seems far preferable than access to information.

Re:A Framework for Moderation by arkh on Wednesday August 07, 2019 @11:04AM (#20635350)
> I don't want a Chinese-style great firewall

But the person you're responding to does. They may not admit it to themselves but that's what they want.

When I see this kind of discussion and how "Internet helps terrorists" I imagine them in a military uniform doing the same speech about the Résistance.

Re:A Framework for Moderation by laughinghan on Wednesday August 07, 2019 @02:22PM (#20637384)
China's Great Firewall is run by the state. By law you're not supposed to circumvent it.

There is no law that says you have to use Cloudflare and if they ban you, you're not allowed to be on the Internet. It just costs you time and effort, possibly money, hopefully reach.

What leads you to believe that the person they're responding to doesn't simply wish more private entities would exercise their legal right to listen to their conscience on who is too hateful to do business with? That's not censorship of important information, that's the free market of ideas at work.

The Stratechery article explicitly discusses this phenomenon and endorses it. What leads you to believe that the person they're responding to disagrees with the article on this point?

Re:A Framework for Moderation by frereubu on Wednesday August 07, 2019 @12:07PM (#20635976)
I used to know the person who ran the Teens website on the BBC around 15 years ago. She was at a conference about protecting children online and spoke to someone from the section of the police who dealt with tracking and apprehending paedophiles who met online. Even back then he had seen online meeting points for paedophiles where the fact that they could speak to each other openly about what they wanted to do normalised what they were talking about, and that had lead to real-world consequences. I think about this often when reflecting on how this kind of depersonalised (in the sense of not being present) hate speech seems to be growing and what it might lead to.
Re:A Framework for Moderation by atoav on Wednesday August 07, 2019 @12:30PM (#20636259)
Sex and violence grab more attention than the cleverest intellectual remark and they usually take less space to be encoded. The design of our internet plattforms is such that these attention grabbers are extremely pushed while valuable statements are drowned.

We on HN should know more than anybody that certain design decisions will nudge communications in certain directions (compare facebook to twitter to hacker news).

They have different goals one could argue and that their design reflects these goals and one wouldn’t be wrong. Only: the goals on the bigger plattforms are more shaped by advertisement and data collection needs than by the valid interests of the users. Similar to the difference between slotmachines and a board game — both let people play, but the former doesn’t has the best for its player in mind.

So when you say this is about the nature about our fellow humans, I’d argue that it is about the nature of the digital Zoos we developed for them.

Re:A Framework for Moderation by skybrian on Wednesday August 07, 2019 @01:24PM (#20636829)
Yes, this is a design issue, but a very basic one. Simple amplifiers are enough to get bad results.

Easy resharing is, by itself, enough to make rumors go viral. Chain letters and all the other crap you see on Snopes depend on nothing more sophisticated than email forwarding.

Active content filtering (spam filters that work) is necessary, but the platforms that largely replace email don't have them.

Re:A Framework for Moderation by swebs on Wednesday August 07, 2019 @11:17AM (#20635467)
>Why does this rub me the wrong way so much?

Because you at least subconsciously understand the real intent behind the message. "We must stamp out wrongthink by eroding your rights"

Re:A Framework for Moderation by buildzr on Wednesday August 07, 2019 @12:14PM (#20636050)
It seems a bit of an oversimplification, but it is the impression I get. Many of the decisions we must make are between those of freedom and security. The security argument is easy, especially after a tragedy, but the slow erosion of rights is only apparent after the fact.

I worry that fear will result in rash decisions, creating widespread censorship abilities and banning anonymizing services that can be used to bypass them.

Re:A Framework for Moderation by abrahamepton on Wednesday August 07, 2019 @12:23PM (#20636170)
No, the notion is more like "the Internet enables a fundamentally different kind of communication than we've grown up with, and we should rethink our assumptions accordingly."

It's just not like other kinds of human communication that we've had for our entire history as a species.

Human psychological evolution is thousands of years behind the technical capabilities we've developed in the last few decades. A bit of thoughtfulness in the governance of those capabilities isn't too much to ask.

Re:A Framework for Moderation by commandlinefan on Wednesday August 07, 2019 @12:27PM (#20636218)
Even if I agreed with your premise that humans can’t be trusted to talk to one another, what makes you think that those same humans can then somehow be trusted to determine when other humans can safely talk to one another?
Re:A Framework for Moderation by abrahamepton on Wednesday August 07, 2019 @01:00PM (#20636562)
The premise is not, even remotely, "humans can't be trusted to talk to one another" but rather, "humans have spent our entire existence as a species talking to each other in a different way than we have begun to in the last two decades."

The alternative to trusting humans is not to just leave ourselves open to the vagaries of the winds, or whatever, but to allow malicious actors to take advantage of an unregulated medium to pursue their own interests. Not sure why anyone should think that's a better outcome.

Re:A Framework for Moderation by antepodius on Thursday August 08, 2019 @04:54AM (#20642862)
If the alternative requires the construction of a colossal bureaucracy that empowers bad actors in a way that causes even more harm.
Re:A Framework for Moderation by chasing on Wednesday August 07, 2019 @01:42PM (#20636982)
It's not the idiots that matter in this case, it's the innocents that the idiots harm because they've been stoked by especially heinous (and factually inaccurate) ideas.
Re:A Framework for Moderation by freen on Wednesday August 07, 2019 @01:45PM (#20637007)
Because you are incorrectly interpreting it: it's not we must protect the world from bad ideas, it's that no one has a responsibility to host ideas that they disagree with.

I don't have to host Nazi speech and you are free to do so if you choose.