This is not a sustainable strategy for society. It is of course impossible for algorithms to parse perceived intent by the lowest common denominator, and attempting to do so is nothing more than defensive legal posturing.

I think that both intent and perception are important, and they are both judged in the more modern understanding of "toxicity". Malintent is a direct "strike". But, lack of malintent doesn't mean that perceived offense should be entirely ignored.

Let's imagine a scenario: as in the example above, an English person says "can I burn a fag" and an American person believes they are casually thinking about hurting a gay man. The English person meant no harm; but, the American person is not at fault for not understanding this. They can complain, and the English person should explain their intent. The American person should learn and remember the meaning of this expression in British English, and the English person will need to remember that this expression may sound offensive to American listeners, and avoid it in such scenarios.

Basically, the important aspect is that a misunderstanding is not always the fault of the listener (which the intent-only model suggests: you took offense, but I didn't mean that, so you're silly for feeling offended). It is of course also not only the fault of the speaker (which the perception-only model would suggest: I felt offended by what you said, so it doesn't matter what you meant, you owe me an apology or more). Both parties are involved in a miscommunication, if indeed there was one.

If you call someone out for "offensive speech" and your offense rests on your own misunderstanding, it is absolutely your fault.

As a white guy wearing a suit, I got out of there without being charged. If I weren’t… I can imagine it could have gone very differently.

Also, “is was part”? So much for being clear.

Unfortunately, we seem to be in a place where bad intent is taken for granted and merits an aggressive reaction rather than placing any responsibility on the offended party to be civil and open-minded in their response to be perceiving offense.

In the case of "burning a fag", our hypothetical American has the option of inquiring about the meaning of an unfamiliar idiom rather than jumping straight to calling out the speaker.

bum is not burn.

FYI, it's combination of difficult-to-discern keming with the default font on HN, and your unfamiliarity with the phrase, but it's actually "BUM A FAG" and not "BURN A FAG".

Incidentally, did you intentionally write keming instead of kerning?

Part of being an adult is understanding that generally, people are not 4chan trolls.

If someone who's not a crackpot says to you "can I burn a gay", even though that seems unambiguous, they've probably just misspoken or are joking.

In real life almost everyone seems to get this, online it's like everything is a strawman.

Well, for one, this only applies if you already know that person is not a crackpot. Secondly, burning someone would indeed be an extreme example, but not all (accidental) offenses are so obviously monstrous. If someone said "I hate fags" (intending to say they don't like cigarettes), this would be more likely to be someone's actual homophobic opinion, depending on culture.

It's all just a bit weird to me. If someone said to me "I hate fags", I'd either ask them to clarify, or change the topic. If they were persistent, then yeah, get rid. It's not a problem that requires instantaneous resolution.

I don't really want to take the discussion in this direction, but note that almost all of that "shift" comes from one side of the political spectrum.

There is no happy medium here, particularly since those seeking to offend and to wolfwhistle to signal their allegiance to other people who hate your race (for example) will try to couch their terms to hide within the shadows of plausible deniability unless they feel completely safe.

It's more a question of how many false positives and false negatives your culture is prepared to tolerate.

I certainly agree with your final conclusion: it depends on what society is willing to tolerate in terms of false positives and in terms of false negatives. And particularly, the balance between those two kinds of errors.

Not to say that there isn't anything wrong with how people have been increasingly incensed in the last few decades, but that's basically what it was like in the 90s and before. Someone was like, "whoa dude what you said wasn't cool" was met with "it was just a joke".

Your reasoning cuts both ways I think.

If you want to be a global publisher of your opinions, you’re going to receive a global response, from people across the political and social gamut — the larger your audience, and the more political/ideological your speech, the more substantial a response.

My dictionary’s definition of “Faustian” has this usage example:

> Modern celebrities enter into a Faustian pact with the general public.

What people seem to want is the ability to enforce a one-sided bargain with the public, under which their own toxicity, opinions, and any public affirmations should never be censored, as long as their opinions are the politically correct ones.

So, give hard core racists the benefit of the doubt when they wolf whistle each other?

Can't see any historical precedent for that going disastrously wrong.

>What people seem to want is ... their own toxicity, opinions, and any public affirmations should never be censored

Some people are hypocrites. News at 11.

Some people also want anti-han Chinese racism to be treated equivalently to anti-Uyghur racism because they choose to be blind to the inherent power differential.

I remember reading a story of the Hutus and the Tutsis. The genocide began with the radio - shows that very deliberate but cautious (& deniable) dehumanization of the other that ramped up over time. Nobody policed these deliberately aggravated ethnic tensions. By the time they were consistently obvious it was a little on the late side.

On the other hand, I've been falsely accused of being a racist before and it undeniably hurt my feelings a little bit.

And to shut up targets - "he/she does not mean that", "he/she is just insecure", "you know how he/she is". If you cant prove intent, you had no grounds to even talk about that that person is doing to you again and again.

E.g. people who object to a statement because it might possibly be offensive to another group, despite members of that other group being present and not being offended. Clearly they are doing this out of the same enjoyment of other people's discomfort that you describe.

That said, it's interesting this has come up during a discussion about how the intent behind what someone said is often either misunderstood or discarded.

You are assuming way too much good faith. For every one that does it in earnest there's probably 10 grifters just taking advantage of the situation for their own personal benefit:

The real reason seems to me like virtue-signalling. They don't care whether they're personally offended, as that would require burning social "points" by rocking the boat but potentially getting no outcome out of it (nobody else might agree). It's "safer" to just suck it up than sacrifice social points especially when you're on the lowest ranks.

They also don't know nor care whether the protected class they are worried about is actually going to be offended. Again, rocking the boat and burning social capital for little personal benefit.

This leaves the argument of earning social capital by virtue signaling among like-minded peers. Pick a common issue you already know is popular within your target "market" (racism, sexism, "diversity & inclusion", etc) and raise the problem. This doesn't risk any social capital, after all, people like you won't disagree (for the same reasons) and people that might be on the fence or would like to add nuance wouldn't want to risk their social capital in fear of being labelled a racist/etc. It also transcends the ranks - if anything, a higher-ranking person in the company would risk much more by speaking out than someone on your rank or lower.

This eventually results in a positive feedback loop where everything will become offensive given enough time and those who disagree either comply or get pushed out. That's how you get bullshit like GitHub's "main" branch which helps nobody while continuing to sell services to ICE which you could argue actually hurts minorities.

And therein lies the problem. A subset of people get a benefit from engaging in this behavior. So of course they do it at every opportunity.

You get what you incentivize.

Until the outcome is so frequently a net negative (however slight) in so many cases that the behavior is marginalized to a negligibly small number of people/settings the behavior will persist in enough volume to be worth caring about.

The problem is that a large part of the tech industry nowadays relies on "engagement" aka monetizing user attention for the benefit of advertisers, thus extremely vulnerable to public opinion (the advertisers themselves would engage in this behavior, thus companies that supply them must also conform to it). The entire industry is on thin ice already so it makes sense for everyone to play it safe even though it just reinforces the feedback loop even more.

This is why this behavior is rarely seen in other industries that make the bulk of their money on a tangible output, both because the output is less fungible/replaceable as well as those industries' employees typically having less time to waste engaging in bullshit.

I'm just going by conversations I've had with people in person and online, and assuming that this is broadly reflective of society overall. Don't get me wrong, I am quite sure that there are people who are cancelling people for sadistic pleasure and get a kick out of torturing someone and use hot-button issues to do so. However I cannot imagine this type of person outnumbering those with good-faith concerns at a 10:1 ratio.

Other times, you get the "well it's so very easy to make this small change, it's practically rude (or suspicious) not to" - a completely wrong sentiment for many reasons, but easy to make you look difficult for arguing about.

Free speech is not without cost, but it has been successfully utilised to everyone's benefit for hundreds of years. I can only surmise that those advocating for this have never read about or understood the importance of the Enlightenment. They don't understand the horror of living under a social regime where the powerful control language and effectively thought. They don't understand why the ability to risk offense is critical for democracy, science, and social progress. Eliminating free speech will harm the most vulnerable in society; not the most powerful.

American history itself had tons and tons of situation where people risked a lot for saying things - including death. All within the scope you talk about. The time span you talk about include runup to civil war, civil war, reconstruction - all of which were full of political violence.

This also when dueling was the thing. Where men of means were expected to shoot each other over words. If you did not, you was effectively over in that town. And the line between "duel worthy" offense and not could be very thin.

> Eliminating free speech will harm the most vulnerable in society; not the most powerful.

Moreover, we are not talking here about legal standard at all. Neither was legal standard EVER. You could legally seek to insult and offend people as much as you wanted. Nobody cared about intent. This is about when people say "you are jerk".

There is threat to free speech, especially with new proposed laws, but it has zero to do with intent vs effect.

You would think we would learn from these examples and make offensive speech more acceptable, rather than less.

I'm not sure how you conflated legality with social acceptance. My arguments center around the later.

Now do European history pre-enlightement.

It was way, way worse. Shit talk the church and the inquisition slows up. Complain about the cut of your grain that the mill takes and the local lord threatens you lest other peasants start complaining.

The acceptance of freedom of speech as a general principal has greatly increased the civility of civilization.

You are certainly correct - unfortunately the people burning the books are not the ones reading or writing them (even if they pretend to do so, with advanced credentials or padded resumes).

- all social media implemented simple word filters. A word is on YOUR list, the post or chat line doesn't make it to your page/chat. It'd be awesome if these lists were exportable, syncable, and tradeable online.

And I wish the AI revolution would benefit me personally instead of companies.

I would like an AI that censors on my behalf only if and when I want it to, and a personalized AI that I can train to flag accounts/people that I don't want to interact with based on language. I would also like some mechanism to indicate how much is getting caught in the AI-generated filters and allow me to stop or modify its behavior. Once the AI is doing mostly what I want, I would like to be able to share it. I don't know how that would work.

- Group owners should have similar mechanisms for all posts in a group.

Another example: faggots are a traditional meatball dish from parts of England and Wales-the term has nothing to do with anyone’s sexuality. And yet, American social media companies have repeatedly blocked British users for posting about the dish.

There is also a British pudding called “spotted dick” - in this context, “dick” is a dialectical term for “pudding”, no known connection with genitalia.

A friend of my wife recently had her Facebook account restricted for making a post about cooking jerk chicken for dinner. Facebook claimed she was using “abusive language”

[1]: https://news.sky.com/story/spotted-dick-renamed-spotted-rich...

Or try explaining panto to Americans. Or even Allo Allo, a show which would .. definitely not be made nowadays.

I've seen false positives referred to as the "Scunthorpe problem". Someone on the internet has a nice map of filthy-sounding real UK place names ...

There's a reason Essex University had to register sx.ac.uk as an alternative domain name.

For starters, you can't unless you actually include the context in the training set.

Then again, a lot of humans won't successfully manage that either...

It's worse than that. The context doesn't just have to be in the training set. It also has to be available on the other end, when you're using the model to make a determination. And it isn't.

Which is why even human censors can't get it right. The parties communicating can, and almost always do, have external shared state that The Decider doesn't. Which gives the same words different meaning.

Imagine hearing an inside joke you're on the outside of and then being asked to adjudicate whether it was offensive.

E.g. if judging tweets for example you'd presumably do a lot better if you evaluated the tweets in context of followers and in context of past tweets - both recent and the accounts history. E.g. an account that posts white supremacy tweets regularly is likely to mean something entirely different if RT'ing a BLM tweet with "that is fucking awesome" than what someone with #BLM in their profile is likely to mean.

Part of the problem is that they're throwing away a huge amount of the state they do in fact have.

But I absolutely agree it's in general an unsolvable issue - I pointed to a story from my childhood elsewhere showing how trivially people cause problems for "The Decider": A childhood friend being forced not to call his brother names and switching to using names of cheeses.

Is "Edam" an insult or a food preference? You can't know without context.

And it needs to be very local context, because humans very quickly pick up on when you just start using a word to mean something else, so it doesn't even need to be any shared external state about the word, just a shared understanding of where the receiver might expect an insult coupled with an unexpected response that will then easily get labelled an insult. That unexpected term might well in itself be positive if the receiver expects criticism. "Awesome" and "I love it!" are perfectly good insults when the other party has just told you something where the appropriate response would be negative, for example.

Must never use followers. Or anything else the account holder has no control over. Otherwise you'll end up with the same kind of SEO problems where competitors and foreign governments create fake accounts to follow or interact with disfavored ones and destroy their algorithmic reputation.

Guilt by association in general is malice. Someone who regularly interacts with white supremacists might be a white supremacist -- and maybe 90% of them are -- but it could also be someone criticizing, mocking or debunking them. And if it gets out that the algorithm is penalizing people for doing that, they'll stop. Which is very bad.

I specifically follow a few locals that I would often notice upset with the same politicians that I was, but for diametrically different reasons. Even though I disagree with them on most everything, I find value in having a few of those voices on my feed visibly attached to a consistent person. It helps me to resist seeing the "other side" as just an impersonal sea of voices.

100%. It's crazy how often context is missing from datasets! We wrote a separate blog post recently about this too: https://www.surgehq.ai/blog/why-context-aware-datasets-are-c...

Obviously also a technical term used all the time in its technical sense with no problem.

Context and culture is way more important than the actual words used when trying to determine the meaning of a statement.

https://www.urbandictionary.com/define.php?term=nonce

"Obviously also a technical term used all the time in its technical sense with no problem."

Or sometimes not ... a blog I'm involved with recently had a WordPress server problem which caused commenters to receive a message like "Illegal operation: bad nonce". The readers are non technical and mostly British. Some had to be calmed down a bit!

Maybe it's an age thing? Everyone my age knows what nonce means but mostly because it was so amusingly satirized by Brass Eye in the late 90s at the height of a media induced paedophilia scare. Chris Morris got politicians and celebrities to say stupid stuff on camera by telling them it was for charity literally called Nonce Sense:

"Did you know that a child under the influence of a paedophile may smell like hammers? That's right - I'm talking Nonce Sense"

https://thesundae.net/2021/01/21/talking-nonce-sense-in-prai...

The program was a sensation at the time but probably only within a certain age group. I can't imagine that sort of satire would have appealed to older people.

There have been some unfortunate mistakes from companies not aware of this. https://www.dailymail.co.uk/news/article-9760601/Cryptocurre...

https://groceries.aldi.co.uk/en-GB/p-mr-brains-6-pork-faggot...

Before I noticed this comment, I made one of my own, making the same point - and it seems HN has marked it as dead for much the same reason: https://news.ycombinator.com/item?id=30069899

‘Tis ironic. (edit: thank you to whoever vouched my comment, it isn't dead any more)

This guy did: https://youtu.be/3-son3EJTrU

Humans are inventing double-entendres to create the ambiguity.

there are enough trigger happy people with less understanding than an AI, all eager to kick enough rages to wear down a saint…

Many business models based on user generated content wouldn't be possible if the businesses had to pay minimum wage to people for moderating that content. Using an AI model, no matter how broken, allows them to seem more concerned than if they were just relying on an old-school word filter without doing any actual due diligence.

This isn't documented anywhere, but I find it hilarious and dystopian that posts that contain the world "hate" in the description or in the comments get blocked more easily, or soft-banned (they don't show up in anyone's feed), even if the word is used in harmless contexts...

Content management done well is expensive, and requires humans with superior grokking skills. No place cares enough to try.

Sure "AI" can do a lot of impressive first-level pattern matching, and that can be the basis of many useful outputs.

But ANYTHING that requires actually understanding the context, whether it is existence of new obstacles for a 'self-driving' vehicle, understanding of actual meaning for a language model, or anything else, is a complete and utter failure.

Despite appearances, while some of it is genuinely useful, we're really no further than fancy parlor tricks. Crack even the next level of contextual understanding, and it will be an astonishing leap.

I've been following some drama in chat/user interaction in modern games. People are getting banned in Forza Horizon 5 for funny decals. One guy got an 8,000 year ban for a Kim Jong-un KFC paint job (https://i2-prod.dailystar.co.uk/incoming/article25686363.ece...).

EA is banning entire accounts - access to ALL games on an account - for swearing in Apex Legends. All of the major studios are in the process of severely restricting and even removing chat in their games.

The latest Battlefield 2042 game doesn't even have voice chat, and they removed the global scoreboard altogether because they didn't want poorly performing players to know how poorly they were performing.

The list goes on and on. To me, this is all a serious reduction in the enjoyment I have in games. I grew up playing Counter Strike, and s**-talking and competing was a big part of the experience. I just won't play games where I'm not allowed to interact with other players.

That being said: I specifically only buy entertainment products that do NOT require me to listen to some 14-year-old describing sexual activity with my female family members in great detail because his nonexistent skills make me keep winning. Different strokes for different blokes.

At some point games companies were bound to notice the ways voice chat limited their market and remove it.

Stop taking features away from the rest of us

Yes, of course everyone realizes that no filter actually works in practice. But they are not built to work, they are built to give the smallest impression of working. Tumblr doesn't need their site to be absolutely clear of nudity to attract investors, they only need to give the impression that it's absolutely clear of nudity and that they are trying to keep it so.