News for hackers. Stuff that matters.

A comment on Why to normalize Unicode strings. Parent comment

Re:Why to normalize Unicode strings by jrochkind1 on Wednesday March 13, 2019 @03:38PM (#19382785)
OK, thanks.

You can certainly have invalid bytes for UTF-8 too. I think in any unicode encoding, not every possible byte sequence is a valid representation of unicode codepoints. It doesn't require UCS-2/UTF-16.

Here's an invalid byte sequence in UTF-8: "\xc3\x28"

[ Parent | on HN ]
Re:Why to normalize Unicode strings by happytoexplain on Wednesday March 13, 2019 @03:43PM (#19382858)
True, though the GP's example was somewhat unique in that UTF-16 created the possibility of invalid combinations at the code point level, rather than only the byte level.
[ Parent | on HN ]