Re:UTF-8 good, UTF-16 bad
by ot on Wednesday February 09, 2011 @06:32AM
(#2197116)
> You shouldn't be using list on raw unicode strings.
Why? I am using list only to show what are the values of s[0] and s[1].
What I am saying is that it returns the list of characters of the underlying representation, so a list of wide chars (possibly surrogate) if compiled with UTF16 or a list of 32bit characters if compiled with UTF16.
Are you suggesting that all the string processing (including iteration) should be done on a str encoded in UTF8 instead of using the native unicode type?
Why? I am using list only to show what are the values of s[0] and s[1].
What I am saying is that it returns the list of characters of the underlying representation, so a list of wide chars (possibly surrogate) if compiled with UTF16 or a list of 32bit characters if compiled with UTF16.
Are you suggesting that all the string processing (including iteration) should be done on a str encoded in UTF8 instead of using the native unicode type?