Unicode was created to unify a number of different pre-existing character sets so they all could be mapped directly to unicode code points. Some of these character-sets had pre-composed characters, e.g. a single code-point to represent 'ö'. Other used combining characters. Unicode therefore had to support both.
Re:Why to normalize Unicode strings
by paulddraper on Wednesday March 13, 2019 @09:20PM
(#19385686)
The Unicode space is massive. It's odd they didn't just spend a bunch of code points for those existing (predominately Latin) character.
Unless you are supposed to be able to put an unlaut on any character. CJK characters with umlauts.
Yes, you’re supposed to: &̈ *̈ ~̈ ⺃ ⺃̈ ⻑ ⻑̈
It wasn't massive back in UCS2 days...
In particular, Unicode codepoints 128-255 were explicitly defined to be the same as ISO-8859-1, which has a bunch of precomposed characters in it.
Re:Why to normalize Unicode strings
by paulddraper on Wednesday March 13, 2019 @09:18PM
(#19385678)
Yes, I am likewise confused.
Text is one of those enormously complicated human conventions, so it's probably just ignorance speaking. But I would like to understand.
As other people have mentioned, it's for historical reasons. If Unicode was started from a clean slate, without the need to be compatible with existing encodings there wouldn't be any precomposed forms.
Naively, that appears as a major defect in Unicode.
Perhaps someone reading this knows why this was the right thing to do?