News for hackers. Stuff that matters.
Re:Are popular toxicity models simply profanity detectors? by Gareth321 on Tuesday January 25, 2022 @03:01AM (#30069375)
I couldn't agree more. I've been watching this shift in acceptable standards in America move from intent to perception with considerable consternation. It seems that an absence of malintent in language is no longer sufficient defense. If one perceives the language as offensive, it is. This is an absurd standard because with a sufficient audience size, someone will find something offensive. This standard is a de facto end to all discussion. It only takes one hyper sensitive audience member (legitimate or troll) to shut down all discussion.

This is not a sustainable strategy for society. It is of course impossible for algorithms to parse perceived intent by the lowest common denominator, and attempting to do so is nothing more than defensive legal posturing.

Re:Are popular toxicity models simply profanity detectors? by simiones on Tuesday January 25, 2022 @07:34AM (#30071081)
> I've been watching this shift in acceptable standards in America move from intent to perception with considerable consternation.

I think that both intent and perception are important, and they are both judged in the more modern understanding of "toxicity". Malintent is a direct "strike". But, lack of malintent doesn't mean that perceived offense should be entirely ignored.

Let's imagine a scenario: as in the example above, an English person says "can I burn a fag" and an American person believes they are casually thinking about hurting a gay man. The English person meant no harm; but, the American person is not at fault for not understanding this. They can complain, and the English person should explain their intent. The American person should learn and remember the meaning of this expression in British English, and the English person will need to remember that this expression may sound offensive to American listeners, and avoid it in such scenarios.

Basically, the important aspect is that a misunderstanding is not always the fault of the listener (which the intent-only model suggests: you took offense, but I didn't mean that, so you're silly for feeling offended). It is of course also not only the fault of the speaker (which the perception-only model would suggest: I felt offended by what you said, so it doesn't matter what you meant, you owe me an apology or more). Both parties are involved in a miscommunication, if indeed there was one.

Re:Are popular toxicity models simply profanity detectors? by slibhb on Tuesday January 25, 2022 @08:53AM (#30071755)
> Let's imagine a scenario: as in the example above, an English person says "can I burn a fag" and an American person believes they are casually thinking about hurting a gay man. The English person meant no harm; but, the American person is not at fault for not understanding this.

If you call someone out for "offensive speech" and your offense rests on your own misunderstanding, it is absolutely your fault.

Re:Are popular toxicity models simply profanity detectors? by madaxe_again on Tuesday January 25, 2022 @09:47AM (#30072311)
Yes, but not necessarily your problem. I’m British. I once asked for a glass of water in a restaurant in Virginia, and while I thought the waitress’s response was a little peculiar (no reply, just turned around and walked away), I was astounded when the police showed up five minutes later. Apparently she’d misheard me, and thought “just a glass of water, please” was a threat. It was never revealed to me what the threat I had apparently made was, but it cost me several extremely stressful hours convincing the local PD that there had been a misunderstanding - so while it was her fault, it was my problem. I’ve puzzled for years over what she heard.

As a white guy wearing a suit, I got out of there without being charged. If I weren’t… I can imagine it could have gone very differently.

Re:Are popular toxicity models simply profanity detectors? by gowld on Tuesday January 25, 2022 @11:17AM (#30073588)
Maybe is was part your fault for not speaking clearly.
Re:Are popular toxicity models simply profanity detectors? by madaxe_again on Tuesday January 25, 2022 @04:33PM (#30078396)
I speak perfectly clearly - in a British accent. If it’s my fault for not being American, then so be it.

Also, “is was part”? So much for being clear.

Re:Are popular toxicity models simply profanity detectors? by mauvehaus on Tuesday January 25, 2022 @09:06AM (#30071892)
The major issue with this model is that the offended person has options that are a bit less hostile and assuming of bad intent than "calling somebody out".

Unfortunately, we seem to be in a place where bad intent is taken for granted and merits an aggressive reaction rather than placing any responsibility on the offended party to be civil and open-minded in their response to be perceiving offense.

In the case of "burning a fag", our hypothetical American has the option of inquiring about the meaning of an unfamiliar idiom rather than jumping straight to calling out the speaker.

Re:Are popular toxicity models simply profanity detectors? by gowld on Tuesday January 25, 2022 @11:15AM (#30073568)
This is offensive to kerning.

bum is not burn.

Re:Are popular toxicity models simply profanity detectors? by kwooding on Wednesday January 26, 2022 @11:46PM (#30096322)
This is, hands down, the greatest HN comment I have ever read.
Re:Are popular toxicity models simply profanity detectors? by shrikant on Tuesday January 25, 2022 @08:23AM (#30071481)
> [...] an English person says "can I burn a fag" and an American person believes they are casually thinking about hurting a gay man.

FYI, it's combination of difficult-to-discern keming with the default font on HN, and your unfamiliarity with the phrase, but it's actually "BUM A FAG" and not "BURN A FAG".

Re:Are popular toxicity models simply profanity detectors? by userbinator on Tuesday January 25, 2022 @03:39PM (#30077701)
It's interesting to see so many people thought it was "burn" --- perhaps the reference to cigarettes primed the "mental autocorrect" in the wrong direction.
Re:Are popular toxicity models simply profanity detectors? by simiones on Tuesday January 25, 2022 @11:44AM (#30074033)
Yes, you're right, thanks for correcting that!

Incidentally, did you intentionally write keming instead of kerning?

Re:Are popular toxicity models simply profanity detectors? by shrikant on Tuesday January 25, 2022 @11:58AM (#30074281)
Re:Are popular toxicity models simply profanity detectors? by throwaway22032 on Tuesday January 25, 2022 @08:59AM (#30071821)
> the American person is not at fault for not understanding this.

Part of being an adult is understanding that generally, people are not 4chan trolls.

If someone who's not a crackpot says to you "can I burn a gay", even though that seems unambiguous, they've probably just misspoken or are joking.

In real life almost everyone seems to get this, online it's like everything is a strawman.

Re:Are popular toxicity models simply profanity detectors? by simiones on Tuesday January 25, 2022 @11:11AM (#30073507)
> If someone who's not a crackpot says to you "can I burn a gay", even though that seems unambiguous, they've probably just misspoken or are joking.

Well, for one, this only applies if you already know that person is not a crackpot. Secondly, burning someone would indeed be an extreme example, but not all (accidental) offenses are so obviously monstrous. If someone said "I hate fags" (intending to say they don't like cigarettes), this would be more likely to be someone's actual homophobic opinion, depending on culture.

Re:Are popular toxicity models simply profanity detectors? by throwaway22032 on Tuesday January 25, 2022 @03:48PM (#30077806)
But, again, most people aren't running about spouting horrible stuff, and neither are people trying to catch them out either.

It's all just a bit weird to me. If someone said to me "I hate fags", I'd either ask them to clarify, or change the topic. If they were persistent, then yeah, get rid. It's not a problem that requires instantaneous resolution.

Re:Are popular toxicity models simply profanity detectors? by userbinator on Tuesday January 25, 2022 @09:19AM (#30072029)
I've been watching this shift in acceptable standards in America move from intent to perception with considerable consternation.

I don't really want to take the discussion in this direction, but note that almost all of that "shift" comes from one side of the political spectrum.

Re:Are popular toxicity models simply profanity detectors? by mbrodersen on Tuesday January 25, 2022 @10:30PM (#30081321)
Yep you are absolutely right. People routinely get offended by anything that they don’t agree with, and often interpret whatever is said using the worst possible out of context straw man interpretation. I personally get offended when PC advocates whip themselves into a frenzy over the slightest misunderstanding. So I guess I should blame those PC advocates for harming me?
Re:Are popular toxicity models simply profanity detectors? by pydry on Tuesday January 25, 2022 @06:54AM (#30070809)
Intent is always difficult and often impossible to judge.

There is no happy medium here, particularly since those seeking to offend and to wolfwhistle to signal their allegiance to other people who hate your race (for example) will try to couch their terms to hide within the shadows of plausible deniability unless they feel completely safe.

It's more a question of how many false positives and false negatives your culture is prepared to tolerate.

Re:Are popular toxicity models simply profanity detectors? by mcherm on Tuesday January 25, 2022 @07:02AM (#30070855)
Of course, offense is just as difficult and impossible to judge as intent. In both cases, one can simply ask the individual, "Were you offended?" or "What was your intent?" But biased or motivated itself reporting makes each unreliable.

I certainly agree with your final conclusion: it depends on what society is willing to tolerate in terms of false positives and in terms of false negatives. And particularly, the balance between those two kinds of errors.

Re:Are popular toxicity models simply profanity detectors? by yeetaccount4 on Tuesday January 25, 2022 @09:20AM (#30072036)
Anyone can say they’re offended without any cost to themselves. It’s ‘cheap’, and the outcome is often very ‘expensive’ for the offender. You could even make the case that virtue is derived from being offended. This disproportionality invites bad faith. I’ve stopped caring about people’s offense.
Re:Are popular toxicity models simply profanity detectors? by zehaeva on Tuesday January 25, 2022 @10:37AM (#30072965)
Anyone can say they didn't mean it that way/they were joking without any cost to themselves as well.

Not to say that there isn't anything wrong with how people have been increasingly incensed in the last few decades, but that's basically what it was like in the 90s and before. Someone was like, "whoa dude what you said wasn't cool" was met with "it was just a joke".

Your reasoning cuts both ways I think.

Re:Are popular toxicity models simply profanity detectors? by yeetaccount4 on Tuesday January 25, 2022 @10:46AM (#30073086)
It doesn’t cost the other party anything to say you were joking. Saying you’re offended can lead to lost economic opportunity for somebody.
Re:Are popular toxicity models simply profanity detectors? by teakettle42 on Tuesday January 25, 2022 @09:55AM (#30072404)
The happy medium, such as it is, is to not try to police “toxicity” in the first place.

If you want to be a global publisher of your opinions, you’re going to receive a global response, from people across the political and social gamut — the larger your audience, and the more political/ideological your speech, the more substantial a response.

My dictionary’s definition of “Faustian” has this usage example:

> Modern celebrities enter into a Faustian pact with the general public.

What people seem to want is the ability to enforce a one-sided bargain with the public, under which their own toxicity, opinions, and any public affirmations should never be censored, as long as their opinions are the politically correct ones.

Re:Are popular toxicity models simply profanity detectors? by mavhc on Tuesday January 25, 2022 @11:06AM (#30073418)
What happens when the trolls create millions of accounts to overwhelm the 90% people with good intentions?
Re:Are popular toxicity models simply profanity detectors? by pydry on Tuesday January 25, 2022 @11:11AM (#30073494)
>The happy medium, such as it is, is to not try to police “toxicity” in the first place.

So, give hard core racists the benefit of the doubt when they wolf whistle each other?

Can't see any historical precedent for that going disastrously wrong.

>What people seem to want is ... their own toxicity, opinions, and any public affirmations should never be censored

Some people are hypocrites. News at 11.

Some people also want anti-han Chinese racism to be treated equivalently to anti-Uyghur racism because they choose to be blind to the inherent power differential.

I remember reading a story of the Hutus and the Tutsis. The genocide began with the radio - shows that very deliberate but cautious (& deniable) dehumanization of the other that ramped up over time. Nobody policed these deliberately aggravated ethnic tensions. By the time they were consistently obvious it was a little on the late side.

On the other hand, I've been falsely accused of being a racist before and it undeniably hurt my feelings a little bit.

Re:Are popular toxicity models simply profanity detectors? by watwut on Tuesday January 25, 2022 @03:27AM (#30069558)
Because "intent" rule was easy to abused and was in fact abused often. Directly by people seeking or enjoying to be "toxic" (in loose meaning). All you have to do is to pretend little bit you are clueless again and again, despite being told again and again.

And to shut up targets - "he/she does not mean that", "he/she is just insecure", "you know how he/she is". If you cant prove intent, you had no grounds to even talk about that that person is doing to you again and again.

Re:Are popular toxicity models simply profanity detectors? by marcus_holmes on Tuesday January 25, 2022 @04:19AM (#30069893)
This is a tough problem, because we are also experiencing people who take offence in a toxic manner.

E.g. people who object to a statement because it might possibly be offensive to another group, despite members of that other group being present and not being offended. Clearly they are doing this out of the same enjoyment of other people's discomfort that you describe.

Re:Are popular toxicity models simply profanity detectors? by smcl on Tuesday January 25, 2022 @05:10AM (#30070188)
I think some people do go a little far, but I don't think everyone's doing this out of some sadistic desire to watch people squirm. More likely they're just a bit too earnest and as a result try to correct others and overcompensate. Most people here have been an annoying college-aged kid at some point with an axe to grind on some specific point, we should remember that.

That said, it's interesting this has come up during a discussion about how the intent behind what someone said is often either misunderstood or discarded.

Re:Are popular toxicity models simply profanity detectors? by Nextgrid on Tuesday January 25, 2022 @07:21AM (#30070976)
> More likely they're just a bit too earnest and as a result try to correct others and overcompensate.

You are assuming way too much good faith. For every one that does it in earnest there's probably 10 grifters just taking advantage of the situation for their own personal benefit:

The real reason seems to me like virtue-signalling. They don't care whether they're personally offended, as that would require burning social "points" by rocking the boat but potentially getting no outcome out of it (nobody else might agree). It's "safer" to just suck it up than sacrifice social points especially when you're on the lowest ranks.

They also don't know nor care whether the protected class they are worried about is actually going to be offended. Again, rocking the boat and burning social capital for little personal benefit.

This leaves the argument of earning social capital by virtue signaling among like-minded peers. Pick a common issue you already know is popular within your target "market" (racism, sexism, "diversity & inclusion", etc) and raise the problem. This doesn't risk any social capital, after all, people like you won't disagree (for the same reasons) and people that might be on the fence or would like to add nuance wouldn't want to risk their social capital in fear of being labelled a racist/etc. It also transcends the ranks - if anything, a higher-ranking person in the company would risk much more by speaking out than someone on your rank or lower.

This eventually results in a positive feedback loop where everything will become offensive given enough time and those who disagree either comply or get pushed out. That's how you get bullshit like GitHub's "main" branch which helps nobody while continuing to sell services to ICE which you could argue actually hurts minorities.

Re:Are popular toxicity models simply profanity detectors? by throwaway0a5e on Tuesday January 25, 2022 @09:37AM (#30072215)
>taking advantage of the situation for their own personal benefit:

And therein lies the problem. A subset of people get a benefit from engaging in this behavior. So of course they do it at every opportunity.

You get what you incentivize.

Until the outcome is so frequently a net negative (however slight) in so many cases that the behavior is marginalized to a negligibly small number of people/settings the behavior will persist in enough volume to be worth caring about.

Re:Are popular toxicity models simply profanity detectors? by Nextgrid on Tuesday January 25, 2022 @10:03AM (#30072520)
> You get what you incentivize.

The problem is that a large part of the tech industry nowadays relies on "engagement" aka monetizing user attention for the benefit of advertisers, thus extremely vulnerable to public opinion (the advertisers themselves would engage in this behavior, thus companies that supply them must also conform to it). The entire industry is on thin ice already so it makes sense for everyone to play it safe even though it just reinforces the feedback loop even more.

This is why this behavior is rarely seen in other industries that make the bulk of their money on a tangible output, both because the output is less fungible/replaceable as well as those industries' employees typically having less time to waste engaging in bullshit.

Re:Are popular toxicity models simply profanity detectors? by smcl on Tuesday January 25, 2022 @01:51PM (#30076185)
> You are assuming way too much good faith

I'm just going by conversations I've had with people in person and online, and assuming that this is broadly reflective of society overall. Don't get me wrong, I am quite sure that there are people who are cancelling people for sadistic pleasure and get a kick out of torturing someone and use hot-button issues to do so. However I cannot imagine this type of person outnumbering those with good-faith concerns at a 10:1 ratio.

Re:Are popular toxicity models simply profanity detectors? by Nextgrid on Tuesday January 25, 2022 @08:38PM (#30080575)
Keep in mind that someone doing it maliciously has every reason to cover it up and pretend it’s legitimate.
Re:Are popular toxicity models simply profanity detectors? by smcl on Wednesday January 26, 2022 @12:29PM (#30088578)
We're probably not going to see eye to eye on this. Honestly it sounds like you've been on the receiving end of this, and I can imagine that being a very frustrating experience so I do sympathise if that is the case. But really the world you're painting is one where you have to walk on eggshells in every conversation because a mob is waiting to pounce if you used the word "retard" as a verb or called someone "Miss" instead of "Ms" or something like that. Perhaps that might happen in the most overly-earnest, try-hard, super-woke group where everyone uses custom, ever-changing neopronouns and you're expected to remember and use every one of them. But I think those sort of people are busy enough with their own internal squabbles to be going out bothering HNers.
Re:Are popular toxicity models simply profanity detectors? by marcus_holmes on Friday January 28, 2022 @11:25AM (#30116751)
Have you ever posted anything on Twitter? It's exactly what you describe.
Re:Are popular toxicity models simply profanity detectors? by marcus_holmes on Wednesday January 26, 2022 @03:35AM (#30083148)
Which is fair enough, and I'd agree with that. But why do we then assume that everyone who says anything that could be construed as offensive is definitely malicious and needs to be cancelled?
Re:Are popular toxicity models simply profanity detectors? by smcl on Wednesday January 26, 2022 @12:16PM (#30088335)
I think it's a little unfair to assume that I would do that
Re:Are popular toxicity models simply profanity detectors? by marcus_holmes on Tuesday January 25, 2022 @05:26AM (#30070265)
I agree, but there seems to be no similar leeway for people who have said offensive things with no intent to be offensive. Any speech that could be construed as offensive is presumed to be malicious. So it seems fair to assume that taking offence needlessly is also malicious.
Re:Are popular toxicity models simply profanity detectors? by Chris2048 on Tuesday January 25, 2022 @06:42AM (#30070734)
There's another aspect too: mysterious third-party echo chambers deciding what's now offensive. The black/whitelist thing is a good example; very often the topic is introduced as if the issue is resolved "we got together and had a discussion, and yes it is offensive" and suddenly everyone not present in that conversation has to accept this verdict.

Other times, you get the "well it's so very easy to make this small change, it's practically rude (or suspicious) not to" - a completely wrong sentiment for many reasons, but easy to make you look difficult for arguing about.

Re:Are popular toxicity models simply profanity detectors? by southerntofu on Tuesday January 25, 2022 @05:38AM (#30070339)
I agree, but the fact that different sensibilities can have very strong/opposite opinions about a situation proves that bots are unsuited to regulate human behavior as a "neutral" judgement, which was the original point :)
Re:Are popular toxicity models simply profanity detectors? by Gareth321 on Tuesday January 25, 2022 @03:38AM (#30069628)
I don't contend that. I'm arguing that the cost of the move to a perception based standard is far higher than the former. I'm arguing that it is, in effect, completely unenforceable, arbitrary, and destructive to not just individuals, but society as a whole.

Free speech is not without cost, but it has been successfully utilised to everyone's benefit for hundreds of years. I can only surmise that those advocating for this have never read about or understood the importance of the Enlightenment. They don't understand the horror of living under a social regime where the powerful control language and effectively thought. They don't understand why the ability to risk offense is critical for democracy, science, and social progress. Eliminating free speech will harm the most vulnerable in society; not the most powerful.

Re:Are popular toxicity models simply profanity detectors? by watwut on Tuesday January 25, 2022 @03:54AM (#30069722)
> Free speech is not without cost, but it has been successfully utilised to everyone's benefit for hundreds of years.

American history itself had tons and tons of situation where people risked a lot for saying things - including death. All within the scope you talk about. The time span you talk about include runup to civil war, civil war, reconstruction - all of which were full of political violence.

This also when dueling was the thing. Where men of means were expected to shoot each other over words. If you did not, you was effectively over in that town. And the line between "duel worthy" offense and not could be very thin.

> Eliminating free speech will harm the most vulnerable in society; not the most powerful.

Moreover, we are not talking here about legal standard at all. Neither was legal standard EVER. You could legally seek to insult and offend people as much as you wanted. Nobody cared about intent. This is about when people say "you are jerk".

There is threat to free speech, especially with new proposed laws, but it has zero to do with intent vs effect.

Re:Are popular toxicity models simply profanity detectors? by Gareth321 on Tuesday January 25, 2022 @04:27AM (#30069946)
>American history itself had tons and tons of situation where people risked a lot for saying things - including death.

You would think we would learn from these examples and make offensive speech more acceptable, rather than less.

I'm not sure how you conflated legality with social acceptance. My arguments center around the later.

Re:Are popular toxicity models simply profanity detectors? by mbg721 on Tuesday January 25, 2022 @08:44AM (#30071677)
I don't know that you can take dueling at face-value; dueling got to the point where you were expected to show up to the duel, but actually shooting someone dead was classless and would have made you a social pariah.
Re:Are popular toxicity models simply profanity detectors? by throwaway0a5e on Tuesday January 25, 2022 @09:52AM (#30072374)
>American history itself had tons and tons of situation where people risked a lot for saying things - including death. All within the scope you talk about. The time span you talk about include runup to civil war, civil war, reconstruction - all of which were full of political violence.

Now do European history pre-enlightement.

It was way, way worse. Shit talk the church and the inquisition slows up. Complain about the cut of your grain that the mill takes and the local lord threatens you lest other peasants start complaining.

The acceptance of freedom of speech as a general principal has greatly increased the civility of civilization.

Re:Are popular toxicity models simply profanity detectors? by forgingahead on Tuesday January 25, 2022 @03:46AM (#30069676)
I can only surmise that those advocating for this have never read about or understood the importance of the Enlightenment. They don't understand the horror of living under a social regime where the powerful control language and effectively thought. They don't understand why the ability to risk offense is critical for democracy, science, and social progress. Eliminating free speech will harm the most vulnerable in society; not the most powerful.

You are certainly correct - unfortunately the people burning the books are not the ones reading or writing them (even if they pretend to do so, with advanced credentials or padded resumes).

Re:Are popular toxicity models simply profanity detectors? by tenebrisalietum on Tuesday January 25, 2022 @07:25AM (#30071010)
Standards based on perception sounds ugly at first, but fundamentally it's an expression that you don't have an automatic entitlement to communicate with me. Should a big external entity like government or companies control that? No. Bu I'm totally fine for accepting complete responsibility for filtering my communications intake. I just wish that ...

- all social media implemented simple word filters. A word is on YOUR list, the post or chat line doesn't make it to your page/chat. It'd be awesome if these lists were exportable, syncable, and tradeable online.

And I wish the AI revolution would benefit me personally instead of companies.

I would like an AI that censors on my behalf only if and when I want it to, and a personalized AI that I can train to flag accounts/people that I don't want to interact with based on language. I would also like some mechanism to indicate how much is getting caught in the AI-generated filters and allow me to stop or modify its behavior. Once the AI is doing mostly what I want, I would like to be able to share it. I don't know how that would work.

- Group owners should have similar mechanisms for all posts in a group.