Re:UTF-8 – "The most elegant hack"
by option_greek on Sunday September 29, 2013 @03:44AM
(#6464455)
How sad. One defective format limits another great format.
We don't see the code point space run out anytime soon this time, even with questionable additions like Emoji. It took us 20 years to get to the current state and character encoding rate has slowed down significantly since the early days (mostly because most things are covered already).
I wondered why you didn't write that "UTF-8 is variable width using 8, 16, 24, 32, 40, or 48 bits per code point", because that's how Prosser/Thompson/Pike's UTF-8 idea was proposed, as shown in the "most elegant hack" article.
And here's the answer from Wikipedia: In November 2003 UTF-8 was restricted by RFC 3629 to end at U+10FFFF, in order to match the constraints of the UTF-16 character encoding. This removed all 5- and 6-byte sequences, and about half of the 4-byte sequences.
Even though they eliminated the 5th and 6th bytes, UTF-8 can still represent all of Unicode's 1,114,112 code points.
I'm mentioning this because others probably are wondering the same thing.