> That sounds great until the scandals of the type "Someone reposted SFW segments of this girl's revenge porn video and Facebook is taking days to remove them every time; since they're reposted every 24h they stay up forever" roll in.

This line of argument seems uncomfortably similar to "we need to ban encryption because terrorists and pedophiles can use it". The fact that bad people can occasionally benefit from something isn't a reason to not give it to the rest of us.

> The fact that bad people can occasionally benefit from something isn't a reason to not give it to the rest of us.

It actually can be. There's a continuum.

At one extreme are things like plans for nuclear or biological weapons. Nobody distributes those because terrorists really want them for horrible ends and they're of low utility, so we do without them.

At the other extreme are things like junk mail - we all just put up with that minor abuse of the system because having a cheap, flexible mail system is way better than not having junk mail.

Like I said, it's a spectrum, and sometimes bad people being able to abuse something is a good reason for everyone to forgo it.

> At one extreme are things like plans for nuclear or biological weapons. Nobody distributes those because terrorists really want them for horrible ends and they're of low utility, so we do without them.

My understanding is that how to build nuclear weapons is relatively well-known, and the only reason that terrorist groups don't have them is that they can't get their hands on the required raw materials.

building a weapon that goes boom from hitting critical mass of uranium/other radioactive element is pretty straight forward. Getting enough uranium is one of the hard parts, but it's not the only hard part. getting a nuke somewhere without being stopped is pretty difficult too. sure you can put it in the back of a pickup and blow it up somewhere, but hiding the radiation of that bomb while you travel/assemble isn't easy. Building a rocket is difficult for nations.

Yes, I believe you're right about that, but I believe it's still the case that no one just distributes the plans. You have to learn engineering and nuclear physics then design your weapon yourself.

The point is, principles are great when you only consider their upsides. I'm not against encryption or for ultra-restrictive automated moderation, I'm just saying it's a trade-off.

In practice we want to pick the course of action that minimizes harm. Having your video demonetized is one kind of harm, having videos of harassment or revenge porn posted online is another.

The kind of people who say "we should never have false positives, ever" are never the people who will have to defend these positions once somebody gets harmed by them and kicks up a scandal.