Re:Why to normalize Unicode strings
by Zarel on Wednesday March 13, 2019 @02:52PM
(#19382147)
I do understand the need for the difference between NFC and NFKC, but in hindsight NFD and NFKD seem entirely unnecessary.
[ Parent
| on HN ]
NFD is useful if you want to do a diacritic-insensitive search.
The problem is that diacritic-insensitive search is locale-dependent, so it doesn't do the right thing anyway.
Specific example: in English, you'd want a search for ‘a’ to find ‘ä’ while this is the entirely wrong thing in Swedish where a and ä are distinct letters.
An English speaker probably wouldn't want a search for ‘i’ to match ‘j’ even though the latter just has an extra hook on the bottom.
That's a very good point. There's still a use for locale-insensitive diacritic-insensitive searches, but you're absolutely right that in most cases you'd want it to be locale-aware and therefore NFD isn't sufficient (though it may still be easier to do this on NFD than NFC).