News for hackers. Stuff that matters.
Re:Why to normalize Unicode strings by WalterBright on Wednesday March 13, 2019 @02:49PM (#19382106)
There shouldn't even be any such thing as normalized strings, i.e. two different Unicode sequences that are supposed to be the same character.
Re:Why to normalize Unicode strings by timw4mail on Wednesday March 13, 2019 @03:12PM (#19382424)
Sounds great in theory...but you've just vastly extended the number of "characters". While text parsing is a nightmare, you also get the flexibility of combining different characters together, such as emojis and skin tone modifiers, which outputs a different display character.

On the other hand, I'm not sure its possible to have a font that represents every valid combination of unicode.

Re:Why to normalize Unicode strings by WalterBright on Wednesday March 13, 2019 @04:15PM (#19383273)
Unicode can simply not recognize combining characters where a special code point exists, and vice versa, on a case by case basis. For example, the ä can remain its special code point, and a¨ can display as a¨. Then software that processes Unicode will become straightforward.
Re:Why to normalize Unicode strings by int_19h on Wednesday March 13, 2019 @09:44PM (#19385833)
That ship has sailed for backwards compatibility reasons.
Re:Why to normalize Unicode strings by lozenge on Wednesday March 13, 2019 @04:00PM (#19383077)
In that case you would lose information when transferring text from non-Unicode encoded files to Unicode encoded files. Not a great way for your standard to take off...