Re:Name: Illegitimate
by SomeCallMeTim on Friday April 27, 2012 @11:35PM
(#3901938)
I don't think that's what he meant. A surrogate pair in UTF-16 is still referring to a single code point. It's a good point that it's a place where people could screw up in dealing with Unicode, but it doesn't match his complaint.
What does match is the fact that you can use a huge number of combining characters [1] to form a single glyph; each combining character and the base are a code point, so in order to figure out how many glyphs there are you have to iterate with the knowledge of what code points are combining characters.
Keep in mind UTF-8 can represent everything in Unicode without the surrogate pairs that make UTF-16 as annoying as it is.
UTF-8 is inherently variable-width, but it's a flat scheme in that one universal mechanism suffices to represent everything you can represent.
UTF-16 appears fixed-width until you realize it isn't, at which point you discover it's segmented and the segmentation scheme is both somewhat complex and likely to be missed entirely in naïve testing.
There's a lot of code out there that works fine with one-char code points but breaks on two-char code points.