Re:The UTF-8-Everywhere Manifesto
by colanderman on Sunday April 29, 2012 @03:27PM
(#3906780)
-1 is not a valid Unicode code point. "All 1s" is not adequately defined without saying how many 1s – and Unicode does not specify a maximum bit width. Even if you said "the maximum Unicode code point", that is not all 1s – it is 0x10FFFF.
[ Parent
| on HN ]
Re:The UTF-8-Everywhere Manifesto
by erichocean on Sunday April 29, 2012 @03:28PM
(#3906786)
That's the entire point of choosing -1 as an "end of sequence" marker for a UTF-8 string when the length is not known up front.
A byte containing all 1s is not valid in any Unicode encoding, so if one appears, you'd know you had hit the end of the string.
This doesn't work well with handling ill-formed sequences.
The length is of course known upfront if not of the whole string but at least of the individual small substring.
Re:The UTF-8-Everywhere Manifesto
by colanderman on Sunday April 29, 2012 @10:58PM
(#3907850)
OK, I thought you meant a code point containing all 1s. Thanks for clearing that up.