News for hackers. Stuff that matters.
Re:The UTF-8-Everywhere Manifesto by raverbashing on Sunday April 29, 2012 @02:35PM (#3906621)
"First of all, if you don't know the encoding, then you don't know the encoding"

True. But as you said, you have to know if it's BE or LE on UTF16. And there are ways to determine that automatically. (or it's on the same platform so it doesn't matter). With "ASCII compatible" codes, you can't.

I guess the main issue to me is that UTF-16 is not "ASCII compatible" so you know it's a different beast altogether.

And don't worry, I'm not assuming UTF-16 is fixed width. One should use the libraries and not try to solve this 'manually'.

About UTF-32 think: CPU registers and operations. Working with bytes is inefficient (even with the benefit of smaller size).

Re:The UTF-8-Everywhere Manifesto by luriel on Sunday April 29, 2012 @02:52PM (#3906654)
> True. But as you said, you have to know if it's BE or LE on UTF16.

Yes, with UTF-16 you need to know not just the encoding, but also the endianness. That makes UTF-16 worse, not better.

UTF-16 is really the worst of all possible worlds: tons of wasted space with all the complexities of variable encoding without fixed endianness.