My goal is to try to inject some awareness or concept of scale and accuracy being demanded. You've had the opportunity to do so, you've neglected that option, instead doubling down on the emotive argument. The data happen to be readily available online for anyone bothering to look.

The problem with both the original comment and your reply here is the denominator is ignored:

- How many accounts exist?

- How many are removed?

- How many of those removals are appealed?

I happened to look up the numbers for Facebook a few days ago when similar complaints were lodged against them. And again: I believe in large part that the power and influence of these platforms requires an extraordinary level of care.

That said, Facebook blocked 99.8% of fake accounts preemptively before they were flagged by user, in 2021Q3. For a total of 1.8 billion accounts removed. Whatever experience ordinary users of the system had of fake accounts was after that removal of 1.8 billion accounts, nearly 8 billion per year.

https://transparency.fb.com/data/community-standards-enforce...

I'm looking up Google's own transparency report based on your comment. The YouTube Community Standards report is here:

https://transparencyreport.google.com/youtube-policy/removal...

The headline number is that in 2021Q3, YouTube removed 2.4 million channels. That's an annualised rate of nearly 20 million, and the quarterly removal rate roughly doubled from the 2020Q4 & 2021Q1 rate begining in 2021Q2. The cadence is 55,000 channels per day, or about 2,300 per hour.

For comparison, manual reviewers at maximum capacity can moderation about 700--800 items in a day, leaving roughly 30 seconds per item, based on the New York Times comment moderation desk.

http://www.nytimes.com/interactive/2016/09/20/insider/approv...

I've discussed content moderation in terms to total personal media capacity a few years ago here:

https://old.reddit.com/r/dredmorbius/comments/53qbx3/estimat...

https://old.reddit.com/r/dredmorbius/comments/7qya12/informa...

Note that we're taking both FB and YT accounts at face value, that the removals were legitimate. Facebook offers information on appeals, Google (so far as I can tell) does not. But at a first approximation the removals seem to be mostly valid.

A huge problem is that a small fraction of a very large number ... is still a large number. A 1% misclassification rate of 20 million channels is 200,000 channels/year. A 99.99% accurate classification is still 2,000 channels year. For Facebook, with 5 billion removals/year, a 99.99% accurate rate is 500,000 erroniously banned profiles.

And the risk profiles are asymmetric. For the platform. 99.99% success is ... well, pretty good. But for each individual who's erroneously classified, the consequences range from a major annoyance to catastrophic.

And lest you think I'm making the case that this is "good enough" or "acceptable losses", I'm not. I think it would be fairly straightforward to make a strong argument for the heavy regulation or break-up of these services strictly on this basis. They have too much control and influence, both at the macro and micro scales, and even near-perfect operations, management, and policies are not nearly good enough.

You might also want to think about your own actions, and those of formal and informal institutions in which you believe and/or rely, and consider what their own failure rates in judgement and adjudication are. How many false arrests, or evictions, or debt collections, or sentencings, or executions are you willing to live with, transacted in the name of society?

> My goal is to try to inject some awareness or concept of scale and accuracy being demanded. You've had the opportunity to do so, you've neglected that option, instead doubling down on the emotive argument.

You're doing so by highjacking a thread that is unrelated to your end goal. I've neglected nothing, because the point I was trying to make is unrelated. You seem to believe that all exploration of the problem (someone loses access to their account due to bad algorithms taking enforcement actions) must be looked at through the lens of how hard the problem is to solve, or the discussion isn't worth having.

It's fine to raise awareness about the implications of scale and accuracy, and that's an interesting topic to examine, but you've gone a few steps too far and seem to believe that one must look at the user impact through this lens, or they shouldn't look at all.

Treat the problem of moderation as a black box for a moment:

1) Content goes in, moderated content comes out with some error rate

2) Based on the moderation result, some action is taken against the content and potentially the user

3) Due to the inherent error rate, incorrect actions will be taken against some content/users

And assume for a moment that the error rate cannot be improved; it's still possible to have a meaningful conversation about #2 and #3 without delving into the seeming impossibility of changing the error rate in #1.

There are myriad angles to explore ranging from: how to best run a manual review program, to how to improve enforcement actions so they have less impact on legitimate users, to improving the appeals process to ensure it doesn't take a HN front page article to get something done.

Someone could choose not to even attempt improving the error rate, or discover that there are diminishing returns to lowering the error rate at some point. These are tangentially related, but still technically orthogonal to improving the enforcement/appeals process.

Of course it's still completely valid to examine #1 directly! There's plenty to explore: reducing false positive rates in algorithms, optimizing the manual review process, etc. But it's silly to cast aspersions on the basis that someone chose to focus on the customer facing problem without acknowledging how hard it is to fix that problem.

It's just as reasonable to talk about how dangerous "self driving" vehicles are today. Of course it's a monumentally difficult problem to solve. But we still need to have meaningful outcome-focused conversations about these cars despite those complexities.

> You might also want to think about your own actions, and those of formal and informal institutions in which you believe and/or rely

I do often think about such things, but today's discussion is about a common issue plaguing the YouTube content ecosystem. There are appropriate places and threads to discuss those topics.

I'm mostly aligned with what you're saying, and do understand how difficult these problems are to solve. I take issue with the idea that folks are somehow involved in some kind of dereliction of duty or low value discussion for not injecting sympathy/awe at how difficult the problem is to solve.

Among other issues, if you want to solve a problem, it helps to accurately describe it. And the problem description here is not accurate, either directly or by its implied senses.

The far more likely case is that 1) false-negative errors are relatively rare and that 2) false-positive errors are large in an absolute sense, because the true positive case is staggeringly huge, resulting in "small fraction of a very large number ... is still a large number."

The reason to consider these facts is simply that they're integral to the problem at hand and an accurate statement of what that problem is.

________________________________________________________________

In your three-item flow suggestions, what happens as the inputs of appropriate and inappropriate content are varied relative to one another? What happens as assessment costs are scaled? How do you account for identity (and more specifically, identity attribution), which I'd argue is actually central to this case. What is the role of reputation? How would you account for a changed reputational assessment, either because of a baseline change in the individual (betrayal and changes of allegiance are utterly fascinating in fiction and history for precisely this meaning), or because of transfer or misappropriation of identity tokens (similarly)?

What's your napkin-maths for a manual-review process? What budget(s) are you going to consider? What alternatives to exhaustive review might present themselves?

Oh, incidentally, in looking at both the FB and YT numbers, one point that stands out to me is that "wisdom of the crowd" utterly fails to rank against automated (and presumably AI-based) content and activity detection. One point absolutely common between FB and YT is that they are not relying on user-based flagging to make assessments, they're very ahead of this. (I suspect that user flags are used to help establish ground-truth, though even that is problematic.)

I tossed out some numbers based on hypothetical false-positive rates. I'd be interested in knowing what such rates you'd be comfortable with.

I also strongly suspect that pursing that line of inquiry is in fact a red herring, though I suggest it Because Reasons. I've a suggested approach in mind, but would be interested to see if you find yourself going there as well.

In the case of self-driving vehicles, one factor which we don't have to consider is the validity of the vehicle itself. That vehicle exists in its environment, and its physical manifestation is a robust assertion of that existence. A chief problem with online profiles is determining which are actual steel-and-rubber, erm, flesh-and-blood humans, and which are mere digital simulacra or sock-puppets. Which is to suggest that online content-moderation has an identity crisis, so to speak.

Self-driving cars would have numerous other issues, but that isn't one of them.

You can't personally control the lens through which people look at problems or discuss ideas. Someone approaching a problem differently than you is not automatically a signal that they are wrong, but that they think differently than you do.

I may have communicated that poorly through the examples I offered, but bottom line: you and I are having conversations about entirely different things.

That sword cuts both ways.

I'm not trying to control. I am trying to shine a light on the issue in an area that seems to be pervasively underilluminated.

You're the one who's lashed out several times now against that. If that's not an attempt at control, I don't know what it is.

If you find yourself in a conversation you're not having, one option is to leave.

Another is to see if that conversation might be one that you would benefit from.

That said, someone's hijacked this thread from a direct discussion of the phenonomenon originally at hand to several levels of meta, and in a markedly uninteresting and unilluminating direction.

Indeed it does. And as I very openly stated, there's a time and place to discuss what you've raised as well! Even on this very post.

> I'm not trying to control.

Numerous times, you admonished me and the other poster for not bringing value to the conversation because it didn't include points that you personally find most important. You effectively communicated "If you don't include what I personally deem to be most relevant and illuminating, you should check yourself".

If that's not trying to control, I don't know what is.

> If you find yourself in a conversation you're not having, one option is to leave.

Another comment that cuts both ways. I should remind you that you joined the conversation you weren't having, only to tell someone else how they should change what they were discussing.

> in a markedly uninteresting and unilluminating direction

Perhaps to you. Again, you seem to only consider the conversation relative to your own point of view, and that continues even in this conclusion.

I do intend to leave this thread, because it’s devolved into something that doesn’t really align with the spirit of HN.

Again, a very peculiar and one-sided definition of "control" here.

I don't think this discussion deserves any further attention from me.

Cheers.