News for hackers. Stuff that matters.
Re:230, or not 230? That is the EARN IT question by reggieband on Wednesday April 08, 2020 @04:59PM (#22817072)
I thought it was interesting when Twitch partners started talking about a Twitch policy that seems to hold the partner responsible for moderating their own chat. That is, if you are a partner and you have community members posting prohibited content into your Twitch chat then you stand to pay the penalty through a ban or losing your partnership. You are forced to moderate your own chat thereby relieving Twitch of having to do so (and presumably giving them some plausible argument that they are enforcing some level of site wide moderation).

It was interesting to see the reactions of these streamers since they aren't typical business people or legal experts. There was quite some debate amongst them about the fairness of the streamer being held responsible for random trolls that entered their chats. When I considered the viewpoint of individuals instead of corporations it did expand my view of responsibility/accountability.

Re:230, or not 230? That is the EARN IT question by AnthonyMouse on Wednesday April 08, 2020 @07:04PM (#22818143)
Indeed. The problem with imposing liability on moderators is that they're already doing about as well as they reasonably can at a job that isn't easy. Nobody wants a platform full of spam and disinformation.

But it's inherently a difficult trade off between heavy-handed censorship that catches too many dolphins in the shark net vs. not catching strictly 100% of the bad stuff. If you start imposing liability on the moderators then it forces the trade off into an all or nothing -- either they give up and stop moderating whatsoever or they have to murder all the dolphins because now a single shark sighting puts them out of business and you can't always tell the difference.

It also eliminates the possibility for different platforms to experiment with making the trade off in different ways. Maybe The New York Times wants to have an editor read every user comment before posting it but Reddit has a stronger commitment to free speech. Shouldn't we have both and let the different readers make their choices? Isn't that better than locking in the same compromised criteria for everybody?

Re:230, or not 230? That is the EARN IT question by admiral33 on Wednesday April 08, 2020 @08:34PM (#22818684)
I think it's simple. The US postal service uses postal inspectors to try and identify packages containing narcotics[0], and yet we don't make them liable for the packages they miss. Any attempt to moderate undesirable content on a website should not then make you liable for the content you miss.

[0]: https://www.uspis.gov/about/what-we-do/

Re:230, or not 230? That is the EARN IT question by vageli on Wednesday April 08, 2020 @08:49PM (#22818765)
> I think it's simple. The US postal service uses postal inspectors to try and identify packages containing narcotics[0], and yet we don't make them liable for the packages they miss. Any attempt to moderate undesirable content on a website should not then make you liable for the content you miss.

> [0]: https://www.uspis.gov/about/what-we-do/

Then what about sites that would use this to their advantage and half ass the whole thing? (fwiw I'm with you, just entertaining the argument)

How do you differentiate between ignorance and malice in response? I guess via emails and memos that would come up in discovery?

Re:230, or not 230? That is the EARN IT question by pas on Thursday April 09, 2020 @03:42PM (#22826390)
The same thing that we do with every kind of organization. We demand adequate controls, incentives, whistleblower protection, paper trail (auditability and accountability for the higher ups), documented internal policies and sticking to them. (And demand that they pay for external audits, mock cases, red team tests, and so on.)

This sort of works. And it can be made very effective. (Basically by criminalizing grossly negligent auditing and spending public money on enforcing it, the whole structure snaps into place eventually.) Currently financial organizations are allowed to be lax, because audits are lax, because the banking license authority doesn't give much of a shit.

Re:230, or not 230? That is the EARN IT question by whatshisface on Wednesday April 08, 2020 @09:07PM (#22818887)
If you're ignorant or malicious the same thing happens: the government kindly informs you that someone is using your website to break the law, and then you can either do something or become clearly guilty of knowingly supporting them. (The government knows you know because they know they told you.)
Re:230, or not 230? That is the EARN IT question by jacobush on Thursday April 09, 2020 @02:23AM (#22820280)
They fix that case swiftly, then go on to ignore new cases?
Re:230, or not 230? That is the EARN IT question by whatshisface on Thursday April 09, 2020 @08:58AM (#22822067)
Law enforcement, typically tasked with enforcing laws, would not ignore the new cases. Individual citizens enforcing laws themselves is nice but overall we don't depend on vigilantes.
Re:230, or not 230? That is the EARN IT question by ethbro on Thursday April 09, 2020 @10:06AM (#22822643)
Parent presumably means if the person in the moderator position ignores new cases.

To which it seems like a fair response would rely more on free speech perspectives.

Essentially, can it be said that the majority of the platform is used for illegal activities? Or are the illegal activities a minority of some other, lawful activities?

I'd want a system where Reddit is in the clear, but CreditCardSkimmersChat is not.

Re:230, or not 230? That is the EARN IT question by AnthonyMouse on Thursday April 09, 2020 @11:12AM (#22823291)
> Parent presumably means if the person in the moderator position ignores new cases.

Which is what they're supposed to do. They're not the police, the police are the police.

> Essentially, can it be said that the majority of the platform is used for illegal activities? Or are the illegal activities a minority of some other, lawful activities?

This doesn't work because it ends up prohibiting the lawful things anyway. You have a platform that promises witch hunts and the innocent victims of witch hunts get evicted from there, and you have a platform that promises no witch hunts and those people go there but so do all the witches and then you get calls to shut it down because there are many witches. You're left with nowhere for the innocent victims of witch hunts to go.

> I'd want a system where Reddit is in the clear, but CreditCardSkimmersChat is not.

They tried this with SESTA and it was a monumental failure. It turns out when you do this, CreditCardSkimmersChat.com goes away and is replaced with CreditCardSkimmersChat.ru and all that does is make law enforcement's job harder.

You want CreditCardSkimmersChat.com to carry on existing, because that's the place you send your agents to camp out in the chat with a logger and execute a warrant to have their ISP capture all their traffic, and investigate anybody who shows up from your jurisdiction, and collect stolen credit card numbers to report to the credit card companies before they can be used for fraudulent purchases.

You don't want to shut it down because once you know it exists it's a honeypot with a previous reputation for not being a honeypot, and all shutting it down will do is cause it to reappear in Russia or on Tor or at a new site you haven't rediscovered yet meanwhile lots more credit card fraud is happening, which only makes law enforcement's job harder and less effective.

It's like finally discovering the phone number of the crime boss and calling the phone company and ordering them to disconnect their phone.

Re:230, or not 230? That is the EARN IT question by ethbro on Thursday April 09, 2020 @09:47PM (#22829124)
> You don't want to shut it down because once you know it exists it's a honeypot

That's supposing law enforcement has viable ways to leverage knowledge of its existence into investigation and prosecution.

In a future where we have end-to-end encryption widely deployed, this isn't a trivial ask.

Criminals are lazy, sure. But write laws for the future we want, not the easiest case scenario.

Re:230, or not 230? That is the EARN IT question by AnthonyMouse on Friday April 10, 2020 @12:00AM (#22829863)
> That's supposing law enforcement has viable ways to leverage knowledge of its existence into investigation and prosecution.

Which they do, because that's their job and they do it all day long.

We're somehow talking about both end to end encryption and moderated public forums, but those are two different things. If you have a public forum where anybody on the forum can read the messages then anybody on the forum can read the messages -- including law enforcement. So they join the forum and start investigating in all the usual ways, and get a warrant to have the ISP upstream from the site start logging its traffic so they can start locating the users and getting warrants to bug their homes etc.

When you have end to end encrypted communications, the only people who have the message are the sender and the receiver. Then there is no dedicated CreditCardSkimmersChat site, they can use any generic secure messaging software for that. But this isn't any more difficult for law enforcement than criminals who communicate in person -- you still have to come by some reasonable suspicion of them to begin with somehow, and once you have you get a warrant to install bugs, which overcomes end to end encryption because then you're collecting data at one of the endpoints.

Re:230, or not 230? That is the EARN IT question by ethbro on Friday April 10, 2020 @02:03AM (#22830411)
> Which they do, because that's their job and they do it all day long.

With varying degrees of success. Let's not pretend an encrypted-everywhere world (which includes e2e, onion routing, and other options) opens up like an oyster at the first sign of a warrant.

What would your opinion be if CreditCardSkimmersChat advertised itself as a place to meet other people involved in credit card skimming? With all conversation taken to e2e chats as soon as two people were introduced

Not constructing a pathological case here. Am really interested in how folks feel about the social responsibilities of serious encryption.

Re:230, or not 230? That is the EARN IT question by zeepzeep on Thursday April 09, 2020 @04:25AM (#22820732)
I will steal this argument! Very good example, thank you.

This is so clear, I hope I'll convince some people.

Re:230, or not 230? That is the EARN IT question by appleflaxen on Thursday April 09, 2020 @05:27AM (#22821000)
That analogy is flawed because USPS doesn't know what's in the packages, but you know exactly what's on your forum.
Re:230, or not 230? That is the EARN IT question by anon3109478190 on Thursday April 09, 2020 @06:11AM (#22821160)
Not if it's end-to-end encrypted, which is exactly the topic here. With this bill it's highly likely that policing will be required of services providing end-to-end encryption too, which then makes that encryption impossible while complying with the "best practices".
Re:230, or not 230? That is the EARN IT question by nybble41 on Thursday April 09, 2020 @05:07PM (#22827212)
Also, even if the content itself is 100% public, there's no way site owners can be expected to assign someone to monitor every discussion in real-time. Even just investigating content flagged by other users can easily turn into a full-time job for a moderate-size service.
Re:230, or not 230? That is the EARN IT question by jrockway on Wednesday April 08, 2020 @09:40PM (#22819051)
Up until very recently, Twitch's moderation tools have been total garbage. I can only assume that they put the chat moderation stuff into their partner contract so they can get rid of undesirable partners easily. "We hate you, and here is a message that your moderators missed so buh bye." I also think that partners are well aware of the possibility of Twitch getting rid of them at any time with no recourse; this is why they heavily advertise their social media, Discord, and YouTube channels. You don't ever want to put all your eggs in one basket. If they have to move, it is likely that some of their audience will follow them.

Overall, I don't think Twitch is using chat moderation as much of a bargaining chip in practice. There are cases of streamers rallying their viewers to abuse other streamers. But in chat, really all you can do is have your bots spam racist messages, ASCII art, or "CUTE BOTS AYAYA CUTE BOTS". Links are mostly banned in practice (not by Twitch but with common extensions), Twitch moderates emotes (aggressively now, even for partners), and whatever hurtful things people say... most people opt out of by watching someone else. All of the stuff that is going to result in legal action tends to happen outside of Twitch (Twitter and Discord are cesspools of drama; read something like /r/OverwatchTMZ for examples).

I guess my point is, I don't think Twitch chat is going to be affected much by any regulations. I suppose if your chat consistently harrasses minors, you could be banned. But it seems very very uncommon even in the extremely toxic gaming community. When it does happen, it's incited by the streamer, and so the specifics of chat don't matter.

Re:230, or not 230? That is the EARN IT question by alec_kendall on Thursday April 09, 2020 @01:27AM (#22820057)
> I also think that partners are well aware of the possibility of Twitch getting rid of them at any time with no recourse; this is why they heavily advertise their social media, Discord, and YouTube channels.

I believe this is just good self-marketing, not paranoia. If you're a content creator in today's age, you're going to be trying to get traction on every major platform. It's like in SEO, make sure all your site pages are linked to each other, aka "Make sure people can access all relevant content". It's a way to increase retention and traction, not a fail-safe for being banned.

Re:230, or not 230? That is the EARN IT question by Hnrobert42 on Thursday April 09, 2020 @02:34AM (#22820326)
It can be both.