News for hackers. Stuff that matters.
Re:UTF-8 good, UTF-16 bad by fedd on Tuesday February 08, 2011 @02:05AM (#2192268)
i guess utf-8 requires more computations while processing strings. if you work with just 2 bytes chars, it (might) work faster

// upvoted all replies, you're right

Re:UTF-8 good, UTF-16 bad by edsrzf on Tuesday February 08, 2011 @02:43AM (#2192317)
This is a common misconception about UTF-8 vs. UTF-16. You're missing two important facts.

1. Most UTF-8 string operations can operate on a byte at a time. You just have realize that functions like strlen will be telling you byte length instead of character length, and this usually doesn't even matter. (It's still important to know.)

2. UTF-16 is still a variable-width encoding. It was originally intended to be fixed-width, but then the Unicode character set grew too large to be represented in 16 bits.

Re:UTF-8 good, UTF-16 bad by fnl on Tuesday February 08, 2011 @10:25AM (#2193126)
If I have no Surrogate Range CPs in a string, it is far easier to work with UTF-16 than UTF-8 at the byte level, because all chars are constant size. For UTF-8 that only applies to ASCII. And SRs characters are extraordinarily rare, while non-ASCII chars are extremely common. So my programs ensure at the entry points the string is UCS-2 compatible, and then all subsequent string manipulations are far less complex to handle than with UTF-8.
Re:UTF-8 good, UTF-16 bad by program on Tuesday February 08, 2011 @03:28AM (#2192380)
UTF-16 was never intended to be a fixed-width encoding and has been created in order to support characters outside the BMP which aren't covered by UCS-2.
Re:UTF-8 good, UTF-16 bad by edsrzf on Tuesday February 08, 2011 @03:10PM (#2194589)
Okay, you're right. I shortened what I was trying to say too much. UTF-16 evolved from UCS-2, so its roots are in a 16-bit, fixed-width encoding.
Re:UTF-8 good, UTF-16 bad by lysium on Tuesday February 08, 2011 @02:43AM (#2192318)
But UTF-16 is variable encoding, too?