Re:Python 3 makes it easier to develop high-quality software
by rspeer on Wednesday November 18, 2015 @04:35PM
(#10590916)
If you're dealing with emoji -- and yes, they are incredibly common -- you'd better not use any Python before 3.3.
Non-BMP characters in 2.x-3.2 give you implementation-specific behavior. It can vary from one system to the next. Some string methods fail on "narrow builds" of Python (the ones that store their characters in UCS-2), while others return different results than they would on a "wide build".
Re:Python 3 makes it easier to develop high-quality software
by Avernar on Wednesday November 18, 2015 @04:48PM
(#10591008)
Not an issue because I don't use the Unicode type. All my data comes in as UTF-8 on which I do a quick validation check. Then it's just handled as bytes all the way to and from my database.
It's non-BMP characters that trigger the conversion to 4 bytes per codepoint. Unfortunately Emoji are non-BMP and are super common.