With Python 2 strings you have to be proactive about it and - always use u"" unless you know what you're doing - segregate non-unicode data to boundaries by decode()ing early and encode()ing late, or you're guaranteed to shoot yourself in the foot sooner or later
but if you do it it's quite smooth sailing.
Also,
#/usr/bin/env python
# -*- encoding: utf-8 -*-
helps a lot.All that just brings you closer to Python 3 anyway and really helps when using 2to3.
incompatible character encodings: UTF-8 and ASCII-8BIT
gets pretty tiresome! Using 1.9.2 at least you have .force_encoding ... feels pretty wrong but sometimes it's just needed to get the job done..http://98.245.80.27/tcpc/OSCON2011/index.html
Specifically the third talk:
http://98.245.80.27/tcpc/OSCON2011/gbu.html http://98.245.80.27/tcpc/OSCON2011/gbu.pdf
Excerpts:
Its String functions like upcase or capitalize won’t even look at
anything but ASCII.
It’s completely missing a whole lot of critical Unicode
functionality:
casemapping & -folding
grapheme support
normalization
collation
text segmentation, &c &c &c.
Every Ruby string carries around its encoding, instead of sanely
unifying into Unicode internally like nearly everything else does.
Also: > baked right in to the language
is not synonymous with "intelligently implemented"Note that I wasn't implying that "half-ass," "partial," and "just plain wrong" necessarily all apply to Python and/or Ruby's implementations. Some may apply to some areas while others may not, and really this extends outside of just Python and Ruby, but I'm trying to stay in context here.