You either die an MVP or live long enough to build content moderation

Re:You either die an MVP or live long enough to build conten... by jart on Tuesday September 28, 2021 @03:38PM (#28686564)

The one built-in to Python will get you most of the way there:

    >>> import unicodedata
    >>> unicodedata.normalize('NFKD', '𝓗℮𝐥1೦𝗵𝗲𝗹𝗹𝗼')
    'H℮l1೦hello'

Obviously it isn't going to remap leetspeak characters like 1 -> l but it covers a lot of cases.

[ Parent | on HN ]

Re:You either die an MVP or live long enough to build conten... by zerocrates on Wednesday September 29, 2021 @12:40AM (#28690372)

Obviously you're saying it doesn't cover everything, but a big thing it's not going to catch beyond leetspeak-type situations is the kinds of thing you (used to) see in internationalized domain spoofing: legitimate non-Latin-script letters that just look the same or nearly the same.

NFKC/NFKD will handle "this is another form of the Latin letter A" type stuff but not "Cyrillic A looks like Latin A."

[ Parent | on HN ]