> 1. Python core dev pretended Python 3 was good and ready by, like, 3.1. It wasn't.

IIRC the messaging from Guido and Python core dev circa PyCon 2008 and 2009 was that it was good enough for people to start thinking about how they might migrate. This resulted in experiments with different approaches to migrations, maintenance of versions for Python 2 and 3, etc, but at that time there wasn't a clear end date and no rush to migrate. It seems like a lot of people misinterpreted what was actually said. Similar misinterpretations seem to happen within the wider internet community around the GIL and perf considerations. That cliff is finally here ten years later after a lot of experience with migrations.

Python 3.1 came out 10 years ago. 3.2 was released in 2011. What does that mean about the complaints since, let’s just say 2015 to be fair?

I heard several people saying that 3.6 was the first Py3 worth transitioning to. Largely for async (I think async/await came to live then). There is this take on performance: https://hackernoon.com/which-is-the-fastest-version-of-pytho...

For me, having run a slew of large Python code bases, 3.3 was the first 'useful' version to upgrade to as it re-enabled u"string" syntax. Without that, making 2/3 compatible code was hard.

3.5 fixed % formatting for bytes, further making 2/3 transitions easier (or harder, depending on your use/abuse of strings vs bytes).

3.6 added the first 'exciting' new feature: f"formatted string {literals}" if you don't have an asynchronous type project that can make good use of async/await.

I do think "worth transitioning" is the final group of people who have the most reason to hold off. If new features are what made it worth transitioning, then it probably was 3.6 with async/await. If you relied a lot of unicode and already had a mature codebase, you'd wait too. If it was a new project you'd probably start with 3 much earlier because of unicode.

Ordered dicts and f-strings are also very sweet!

I expect sametmax was referring to the new feature in Python 3.6 that plain old `dict` now preserves the insertion order of its elements. See docs here: https://docs.python.org/3/library/stdtypes.html#dict.values

(Or in language-lawyer terms: CPython 3.6 introduced this behavior and documented it as an implementation detail, and for Python 3.7+ it's guaranteed a feature of the language proper.)

And optional type specifiers. This was the key for me.

The Py3 string handling was beyond frustrating prior to 3.6. For what I do, Unicode complexity is not required. Having to stuff it into everything was more trouble than it was worth.

The ubiquity of emoji alone mean that Unicode is everyone's problem in 2010+, and ignoring it won't make it go away. It's Python 2.x where dealing with Unicode (which is everywhere) is far too complex, and more trouble than it is worth. The "complexity" of Python 3 string handling is worth it, and no worse (even < 3.6) than any other modern programming language, and possibly easier than some by making clear runtime errors where accidents are easiest to make. (Though arguably, perhaps I'm biased by having done plenty of Unicode work in other languages that Python 3 did seem so familiar and easy from the start for my Unicode needs. Your mileage of course varies, we all have different backgrounds.)

Python 3's Unicode handling is uniquely bad. I haven't heard of any other language where you can obtain magic strings that crash the program if you try to print them:

    % python3.7 -c "import sys; print(sys.argv[1])" "$(echo -e '\xff')"           
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    UnicodeEncodeError: 'utf-8' codec can't encode character '\udcff' in position 0: surrogates not allowed

Oh, that's a fun example. ("Fun", anyway.)

The key surprising thing that's going on here is this clever hack (clever, but a hack):

> In Python, file names, command line arguments, and environment variables are represented using the string type. On some systems, decoding these strings to and from bytes is necessary before passing them to the operating system. Python uses the file system encoding to perform this conversion ... > > On some systems, conversion using the file system encoding may fail. In this case, Python uses the surrogateescape encoding error handler, which means that undecodable bytes are replaced by a Unicode character U+DCxx on decoding, and these are again translated to the original byte on encoding.

https://docs.python.org/3/library/os.html#file-names-command...

This is meant as a way of fudging the fact that (a) for UI purposes, you want to treat filenames as text strings; (b) your Linux filenames are probably all encoded as UTF-8 (or your locale encoding); (c) but they might not be -- they could be arbitrary bytes, except only NUL; (d) and if they are, you really want to not munge the name when you go back and try to operate on the file.

The fudge is that filenames get decoded as (by default) UTF-8... but if invalid, the offending bytes get stuffed into the UTF-16 surrogate space. Then filesystem APIs encode as UTF-8, except they look for that surrogate hack and turn those to the original bytes, so it all round-trips.

It goes pretty wrong if you try to hand such a hacked-up string to something that just expects to encode normal real Unicode with UTF-8, though. That's what's happening in your example.

The magic words are `os.fsdecode` -- that's how you get back bytes round-trip clean from that hack.

what an excellent writeup of why modern languages have something like 4 stringy types: bytes, unicode, os string, path string. you could go further if you want to talk to other computers.

That seems more like a difference of opinion on how to handle a bad string than an example of python 3 being bad. I'd argue failing silently is worse, as in programs outside of your toy example you might end up storing badly encoded data etc. Broken data is way worse to deal with than a crashing app

Nothing I do in my ordinary routine full-time engineering work for the past decade has required or does require anything outside the ASCII character set or, alternatively, plain bytes. The use cases my users present to me does not include emoji or non-ASCII characters. If those arise, naturally, one would select tools to address those needs. I don't know the future. In a year or two, my routine work may, of course, change.

> Python core dev pretended Python 3 was good and ready by, like, 3.1. It wasn't.

I would expect any stable release of software to be "good and ready." Can you explain what was wrong with Python 3.0 and 3.1?

> core dev did not listen to any of the problems people pointed out for _years_

What problems were those?

A major, breaking issue[1] was how Python 3 handled missing LC_* variables. By default, a Python script would simply crash with any non-ASCII input or output. It was fixed[2] in 3.5[3].

[1]: https://bugs.python.org/issue19977

[2]: https://www.python.org/dev/peps/pep-0540/

[3]: https://docs.python.org/release/3.5.0/whatsnew/3.5.html

There were very few new features, only breaking changes. With the exception of Unicode defaults which few yet understood.

Strings in python3 were terrible for memory well into mid-3. Raymond Hettinger is a core dev and he recently stated on Twitter that he didn’t feel py3 was a clear upgrade win until this most current release.

Raymond has been consistent in saying Python 3.6.1 was the release where Python 3 became better than Python 2. That was March 21, 2017, and we're now in Python 3.7.4.

I appreciate the general point that that's far more recent than Python 3s initial release but it's also a large exaggeration to say "until this most current release".

Also to understand why it took so long is because Python 2 kept improving after Python 3s release. Python 2.7 was a great release and many further features from Python 3 got backported in it's 2.7.x point release.

Sorry, I misremembered. 3.8 is the first one he said is a clear winner: https://twitter.com/raymondh/status/1167218640095043586

2017 is a lot more recent than 2008 (Py 3.0).

> 1. Python core dev pretended Python 3 was good and ready by, like, 3.1. It wasn't.

The post you're responding to starts verbatim with `Python 2 to 3 (at least by 3.3 or so)` and your reply is about Python 3.1, released 3 years earlier. The amount of changes was quite big.

You've got to let go of these grudges. Time has past, enough time that you shouldn't be complaining about these things here, because all you're doing is giving the Python 2 "out of my cold dead hands" crew more ammo.

Just let it go...