Ask HN: Scientific Study of HN flag dynamics
Would there be any interest in a scientific study of the effectiveness of HN flagging, including:
- false positives
- false negatives
- brigading
- flagged topics vs off-HN contexts
- flagged topic trends over extended time
- content origin (domain, geo, org, author)
Pointers to prior research on distributed flagging and moderation would be welcome.
(
15 comments
| 7 points
| on HN
)
> The open data would tell an incomplete picture. It would still be an interesting to see what you can pull out, but you'd be guessing why a lot of the time.
The current implementation is like the DMCA. Anything can be censored for any reason without input from HN moderators. If the submitter is aware of the process, they can appeal to HN moderators. If HN readers saw the submission before it was flagged, they can also appeal to HN moderators.
If there were organized efforts to censor specific topics, one mitigation could be to deny the benefits of censorship. This could be done from public data by extracting a queue of flagged submissions and feeding them into another level of non-public analysis, automated + human. The surviving stories could then be posted to a dedicated site, e.g. using https://github.com/lobsters/lobsters.
If there were organized efforts to censor specific topics, a secondary public log of valid-but-censored articles could trigger the Streisand effect. Not only could this deny the benefit of censorship, it could cause censorship to have the opposite effect, reducing demand for organized flagging. This would have no impact on organic, non-coordinated flagging.
Flagging certainly exists. What is your evidence in favor of censorship existing here?
If memory serves, there are website(s) that track which submission get flagged. If you’re concerned about a favored view or topic being flagged frequently, you could start there.
Otherwise you risk exerting lots of effort to simply find out that a favored view or topic isn’t shared by lots of people here. And, well, not all of them are equally important, or equally important to the HN audience.
HN has non-public mechanisms to detect some coordinated activity (e.g. upvoting submissions) across registered usernames.
> If there were organized efforts to censor specific topics
Note the "if" at the beginning of the sentence. It's not about transient topics, but long-term patterns. Broadly speaking, "censorship" is only one possible motive for coordinated activity, flagging or otherwise.
> If memory serves, there are website(s) that track which submission get flagged.
Thanks for the pointer, found https://news.ycombinator.com/item?id=29144989 & https://news.ycombinator.com/item?id=29113079 & https://github.com/MoserMichael/flagged-hn.
Add to the facets: the submitted domain. Maybe posts by the armchair experts and quarterbacks on Substack get flagged more often than the WSJ, for example.
In a study, a list of false positive flags could be derived from public data.
In a study, additional false positive candidates could be identified after manual review.
It is very rare for those to be “deflagged.” Rare enough that there’s a moderation comment about turning flags off.
And an appeal that the community be on its better behavior. They are editorial exceptions not rule following corrections.
For the purpose of a study, gravity can be inferred from story rank.
> Sometimes, but only rarely, to the point of flagged-to-dead.
A study could examine the three categories of:
It would need to be long enough that all of those categories were non-zero.HN’s not a court of law.
In this sub-thread, "unfair" is quoted directly from the HN guidelines as the term for the objective event of a reversed flag, i.e. a "false positive", which we are discussing. It's not being used here as an adjective, it's being used to connect this discussion to the official HN FAQ.
Any interesting results could be investigated further, privately, by HN moderators.