Re:A Programmer’s Introduction to Unicode
by jcranmer on Saturday March 04, 2017 @01:16PM
(#13792108)
Every 1-byte charset that uses high bits (i.e., not UTF-7, ASCII, or EBCDIC) has this property. Depending on how you feel about whether unpaired surrogates constitute a valid decoding, UTF-16 has that property as well.
It's not a very good property to have. UTF-8's invalid decodings means that it's very easy to detect if you're not using UTF-8: iconv's charset detector will rule certainty of UTF-8 if it sees just 3 valid UTF-8 multibyte sequences and no invalid sequences, and that's pretty much the only reliable detector.
It's not a very good property to have. UTF-8's invalid decodings means that it's very easy to detect if you're not using UTF-8: iconv's charset detector will rule certainty of UTF-8 if it sees just 3 valid UTF-8 multibyte sequences and no invalid sequences, and that's pretty much the only reliable detector.