News for hackers. Stuff that matters.

A comment on Text normalization in Go.

Re:Text normalization in Go by argon81 on Wednesday November 27, 2013 @01:51AM (#6806858)
I actually took this a step further a few months back and implemented unicode's "Skeleton" algorithm https://github.com/mtibben/tr39-confusables-go

This is useful for example, to ensure that users don't try and spoof each other's usernames. Simply create and store a skeleton string for each username, and keep a unique constraint on it

[ Parent | on HN ]
Re:Text normalization in Go by zellyn on Wednesday November 27, 2013 @02:23AM (#6806950)
That confusables list is a good starting point, although you'll need to make additions, and probably scale back a couple of the over-zealous ones (eg. rn -> m)

I'm coming at this from a comment spam point of view, not usernames, btw.

[ Parent | on HN ]