News for hackers. Stuff that matters.
Re:Mercurial’s journey to and reflections on Python 3 by masklinn on Monday January 13, 2020 @04:17PM (#22038685)
> You just need to be aware that in some cases the work is already done for you by the language, for example in python if you open a file (without "b" option, the python will do the translation on the fly and you don't need to worry about it)

Sadly they fucked up that part rather thoroughly, because the default encoding is `locale.getpreferredencoding()`, which ensures it's going to be wrong at the least possible convenient time and on the devices least accessible for debugging.

Do not ever use text-mode `open` without specifying an encoding.

Re:Mercurial’s journey to and reflections on Python 3 by mintplant on Monday January 13, 2020 @09:06PM (#22041144)
Node.js tries to be helpful in defaulting file writes to UTF-8, but defaults file reads to returning a raw byte buffer [0]. So you have to either remember to treat the two operations differently, or, like in Python, manually specify the encoding for both.

[0] I seem to recall that it used to default to the locale's preferred encoding, but I could have my wires crossed with other languages' standard libraries there.

Re:Mercurial’s journey to and reflections on Python 3 by takeda on Tuesday January 14, 2020 @01:03AM (#22042259)
The locales are provided by LANG and other locate variables, so Python will use whatever is set in environment, you can also specify the encoding in one of open() parameters.
Re:Mercurial’s journey to and reflections on Python 3 by masklinn on Tuesday January 14, 2020 @01:15AM (#22042307)
> The locales are provided by LANG and other locate variables

Which is absolutely not what you want when, say, opening your own data files. Even when opening the user’s files it’s likely not what you want.

> you can also specify …

And what I’m saying is this is not a “can also” it’s a “must”. Not doing so will bite you in the ass, because “whatever random garbage is on the machine” is really not what you want a default to be.

Re:Mercurial’s journey to and reflections on Python 3 by takeda on Tuesday January 14, 2020 @01:44PM (#22047892)
Oh I see your point. Looks like they changed the behavior in 3.7 (they added -X UTF-8 option), but being able to set it from the application would be great.