News for hackers. Stuff that matters.
Re:UTF-8 good, UTF-16 bad by fedd on Tuesday February 08, 2011 @02:02AM (#2192260)
if you want to deal with characters with high numbers you should know code points stuff. for example, the String.length() would return a number of two-bytes chars, not real four bytes characters, which may confuse someone

//edit: this is about Java

Re:UTF-8 good, UTF-16 bad by btilly on Tuesday February 08, 2011 @03:10AM (#2192356)
Exactly. A Java char is not synonymous with a Unicode code point. But the majority of the time they are synonymous, older documentation claimed that they were the same, and this is the meme that many Java programmers (in my experience) have.
Re:UTF-8 good, UTF-16 bad by fedd on Tuesday February 08, 2011 @03:18AM (#2192364)
yes. i write my java-based matrix to be code-points aware so that no-one in Japan and China using it would face any problems.
Re:UTF-8 good, UTF-16 bad by Locke1689 on Tuesday February 08, 2011 @03:09AM (#2192355)
That's actually my point. Python supports Unicode code points and UTF. If you get the output encoding in UTF-8 it would actually be variable length chars. What's important is your coding output, not the internal code point representation.