Why to normalize Unicode strings

Re:Why to normalize Unicode strings by jrochkind1 on Wednesday March 13, 2019 @04:54PM (#19383651)

Strangely, RFC 6531 gives the impression of being written by people who didn't realize unicode normalization was a concern. I'm reading it for the first time, but it basically just says "an SMTP server can advertise it supports UTF8, and then it should accept UTF8 in all the places that were ascii before." I mean, a little bit more complicated than that, but with regard to _mailboxes_ (the part before the `@`), not much.

It doens't seem to say anything about normalization, or the fact that a mailbox `josé@` could be using 'LATIN SMALL LETTER E WITH ACUTE' or 'LATIN SMALL LETTER E'+'COMBINING ACUTE ACCENT'.

I suspect any SMTP servers supporting UTF8 are actualy doing NFC normalization to match mailboxes to requests.

[ Parent | on HN ]

Re:Why to normalize Unicode strings by marcosdumay on Wednesday March 13, 2019 @10:22PM (#19386023)

Servers are free to do any transformation on the account names they accept, and shouldn't apply any on the ones they relay.

It's expected that they would do many kinds of transformations, like case normalization. Unicode normalization should be one of those but none of the email RFCs intend to enumerate them all.

[ Parent | on HN ]