News for hackers. Stuff that matters.
Re:Scunthorpe Sans – a self-censoring font by tzs on Sunday May 10, 2020 @08:50PM (#23137791)
I had occasion to write a chat server for a small gaming service once. The way I came up with to do the profanity filter ended up working a lot better than I expected it to, and avoided the problem you describe.

It worked like this.

1. Make a copy of the text to be filtered, modified as follows.

-- change upper case letters to lower case

-- change 0 to o, 1 to i, 3 to e, 5 to s, and vowels with various accent marks to the corresponding unaccented lower case vowel

-- discard anything else

For example, if the input was

  start the assault. let's f-u-c-k them
it would become

  starttheassaultletsfuckthem 
2. Scan this for any substrings that are on the bad word list. In this case it would find "ass" and "fuck" [1].

3. If no bad words are found, the original string is returned and we are done.

4. For each word in the original string, look it up in the dictionary (I believe I used /usr/dict/words). Note the ranges of character positions in the original string of the words that are spelled correctly.

5. For each bad word found in the filtered string, if all of its characters came from positions in the original string that were part of correctly spelled words, leave it alone. Otherwise, asterisk out its characters in the original string.

In the above example, "ass" would be left alone because all of its characters came from "assualt" which is recognized as a correctly spelled word. None of "fuck" came from correctly spelled words, so it would get zapped. The final result would be

  start the assault. let's *-*-*-* them
[1] Well...actually not. I just checked my archives and found the chat server bad word list. It did not include "ass".
Re:Scunthorpe Sans – a self-censoring font by woodrowbarlow on Monday May 11, 2020 @07:13AM (#23140891)
just... be sure you're at least aware of the scunthorpe problem that this font is named after. do not implement an obscenity filter without being aware of the inconveniences they pose.

my name is woody (like the tom hanks cowboy, but also, i've been told, like the outdated euphemism for an erection) and i've been scunthorped out of creating accounts on dozens of sites because of my name. it's aggravating at the best of times, but doubly so when i want to be actually recognizable on the service in question.

https://en.wikipedia.org/wiki/Scunthorpe_problem