Re:How to Use UTF-8 with Python
by joeyespo on Friday December 02, 2011 @01:49PM
(#3305140)
Python 2 is still the status quo though.
One of my favorite things about Python 3 is its new string handling. It is fantastic and marginally more intuitive. Having an explicit (and a more general) immutable `bytes` class, no longer sharing a base class with `unicode`, and having Unicode instead of a potentially-encoded string as the default are all wonderful decisions and moves Python forward by miles in my opinion.
Until Python 3 becomes the norm, we're still stuck with the confusing `str` and `unicode` constructs.
Re:How to Use UTF-8 with Python
by js2 on Friday December 02, 2011 @12:31PM
(#3304795)
For decoding a byte string, I prefer bytes.decode("utf8"). Note that decode takes an optional second argument that says what to do if an invalid byte sequence is encountered - by default an exception is raised.
Re:How to Use UTF-8 with Python
by oconnore on Friday December 02, 2011 @02:16PM
(#3305251)
$ python3
Python 3.2.2 (default, Nov 21 2011, 05:01:42)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more
information.
>>> print('hello world') # <-- Unicode!
Re:How to Use UTF-8 with Python
by jrockway on Friday December 02, 2011 @12:59PM
(#3304925)
First, you can place a UTF-8 byte-order marker at the beginning of your file, if your editor supports it. Secondly, you can place the following special comment in the first or second lines of your script:
# -- coding: utf-8 --
Does Python actually parse comments, or is this just to get your editor to do the right thing?
Re:How to Use UTF-8 with Python
by js2 on Friday December 02, 2011 @01:12PM
(#3304987)
Python supports writing Unicode literals in any encoding, but you have to declare the encoding being used. This is done by including a special comment as either the first or second line of the source file:
# -*- coding: latin-1 -*-
The syntax is inspired by Emacs’s notation for specifying variables local to a file. Emacs supports many different variables, but Python only supports ‘coding’. The dash-splat-dash symbols indicate to Emacs that the comment is special; they have no significance to Python but are a convention. Python looks for coding: name or coding=name in the comment.
[1] http://docs.python.org/release/3.0.1/whatsnew/3.0.html#text-... [2] http://docs.python.org/release/3.0.1/howto/unicode.html
One of my favorite things about Python 3 is its new string handling. It is fantastic and marginally more intuitive. Having an explicit (and a more general) immutable `bytes` class, no longer sharing a base class with `unicode`, and having Unicode instead of a potentially-encoded string as the default are all wonderful decisions and moves Python forward by miles in my opinion.
Until Python 3 becomes the norm, we're still stuck with the confusing `str` and `unicode` constructs.
Also, I think http://docs.python.org/howto/unicode.html may be a better reference than this article.
# -- coding: utf-8 --
Does Python actually parse comments, or is this just to get your editor to do the right thing?
http://docs.python.org/howto/unicode.html
See: http://www.python.org/dev/peps/pep-0263/