News for hackers. Stuff that matters.
Re:The Fundamental Problem in Python 3 by slavik81 on Friday December 20, 2019 @02:45AM (#21842346)
> Python 3 forces UTF-8 everywhere

If only. My problems with Python 3 were due to things like crashing when piping stdout to a file because for some reason the Python-chosen encoding for that was ASCII-only.

Another was discovering that the Python 3 default encoding for text files on Windows was CP-1252, so it would crash outputting any non-English text to a file.

It's a little better now, but the defaults were awful when I first started using Python 3. I expected it to be Unicode by default. It was not.

Re:The Fundamental Problem in Python 3 by bildung on Friday December 20, 2019 @02:57AM (#21842394)
Yes, I omitted that my Python 2 (mostly Django or bottle) ran on Linux with UTF-8. I suspect your piping case could be due to either the OS not using UTF-8 as locale or the application behind the pipe ignoring the OS' locale.

These encoding problems will probably stay with us for quite a while, regardless of the language we write our applications in.

Re:The Fundamental Problem in Python 3 by slavik81 on Friday December 20, 2019 @03:18AM (#21842465)
Just to elaborate, on a fresh Windows 7 install with en_CA locale, this was the behaviour:

  python3 -c 'print("é")'
  é
  python3 -c 'print("é")' > out.txt
  (UnicodeEncodeError) 'ascii' codec can't encode character '\xe9' in position 1: ordinal not in range(128)
I felt that was unacceptable behaviour for my program, but it was beyond my ability to fix. Though, I think this has been addressed in more recent Python releases.
Re:The Fundamental Problem in Python 3 by mehrdadn on Friday December 20, 2019 @03:31AM (#21842527)
I thought you might have fun with this -- try running my example here: https://news.ycombinator.com/item?id=21842471