Re:Why to normalize Unicode strings
by WalterBright on Wednesday March 13, 2019 @02:49PM
(#19382106)
There shouldn't even be any such thing as normalized strings, i.e. two different Unicode sequences that are supposed to be the same character.
[ Parent
| on HN ]
Sounds great in theory...but you've just vastly extended the number of "characters". While text parsing is a nightmare, you also get the flexibility of combining different characters together, such as emojis and skin tone modifiers, which outputs a different display character.
On the other hand, I'm not sure its possible to have a font that represents every valid combination of unicode.
Re:Why to normalize Unicode strings
by WalterBright on Wednesday March 13, 2019 @04:15PM
(#19383273)
Unicode can simply not recognize combining characters where a special code point exists, and vice versa, on a case by case basis. For example, the ä can remain its special code point, and a¨ can display as a¨. Then software that processes Unicode will become straightforward.
That ship has sailed for backwards compatibility reasons.