This issue is largely OS X related rather than Python-related per se. OS X ships with vendored copies of many common Python libraries which you cannot update, and so if you need to use newer libraries you have to install them in some other location and fiddle with the path to make sure your versions are loaded before the OS X versions. Because the versions included in OS X (including the version of the Python runtime) are pretty old almost everyone ends up doing this.

In theory homebrew takes care of the whole thing for you but, in practice... as an only part-time OS X user I have run into a lot of problems with the path getting set in various ways that end up in various copies of the Python interpreter or library paths getting loaded, resulting in runtime errors due to library versions being inconsistent with the interpreter version or expected libraries not being loaded...

A further complication is that, the specific way that Apple vendored a set of python packages, it is possible but not particularly easy to add new packages to the OS-provided Python runtime, so most users, if they want some package that Apple didn't include, seem to end up installing a whole new environment just to get that package, even if the OS-provided environment were otherwise acceptable to them.

It's a very real problem and very frustrating, but I'm not quite willing to blame it on the Python ecosystem because it seems to be pretty much a result of Apple's decision to ship an interpreter and set of libraries that most users will end up being unhappy with. Anecdotally at least I run into these issues a lot less on Linux because even with fairly stable distributions (e.g. RHEL7.5/8) the included interpreter and library are more recent and the package repositories include most of the Python packages you would want, maintained to match the OS interpreter version. So while some people do end up using virtual environments it's less necessary than it is on OS X, where it frankly seems near impossible to do anything without installing a whole second Python environment and munging paths.

An extra complication is that, as a casual OS X user, it seems to me like OS X terminal emulators and tools are more likely to munge with the path than Linux terminal emulators and tools, so even if you get everything set up "just right" you end up having it not work properly in certain contexts like plugins because something has modified the path for some reason.

Edited to expand that, I guess what I mean to say but failed to articulate, is that the fundamental problem here (in my opinion) is that OS X ships with a Python distribution but not with a package manager. That's the fundamental difference from Linux where these problems are much less common. Third-party package managers like Homebrew and Macports are used, and in order to have full control of the Python environment they have to install a whole second copy because they are not allowed to tamper with the OS X Python environment. This pretty inevitably leads to frustration. The best solution would seem to be for Apple to provide some kind of "first-party-ish" package management functionality, perhaps something like Chocalatey which is not a Microsoft product but is built on Windows-provided primitives, so that the package managers that users are actually using can play nicely with the OS X Python environment.

An extra complication is that, as a casual OS X user, it seems to me like OS X terminal emulators and tools are more likely to munge with the path

This might be a function of the casualness rather than the tools. Here's a thing to check out on OS X:

    man path_helper 

I didn't know about this for ages and it makes your path life much saner - at least you'll have some idea where seemingly magical paths come from. Personally, I use it for everything and don't let other profile scripts eff with the global path at all.

You set up your personal shell paths by adding files to /etc/paths.d/?

With neither shame nor ruth. Is there some non-aesthetic reason not to?

No! Well other than not having your whole shell environment in your dotfiles. It just wouldn’t have occurred to me.

Linux has often had the same issue. The problem is the OS relies on system installed Python and package versions. If you, as a user, upgrade Python or those packages the OS may not gracefully handle them. In Linux you can end up with an unbootable OS. I'm not sure if you could blame Python devs or the OS devs (or the user).

Homebrew can help with distinction, letting the user install to `/usr/local/bin/python` while the OS continues to use `/usr/bin/python`. My personal solution is to install Python packages into my homedir (--user) or use virtualenvs. This also helps when six months from now you want a "clean" setup.

I've pushed any organization I've worked at who heavily relies on Python to decouple Python from the OS because of this and so OS updates can be independent of Python updates.

> An extra complication is that, as a casual OS X user, it seems to me like OS X terminal emulators and tools are more likely to munge with the path than Linux terminal emulators and tools, so even if you get everything set up "just right" you end up having it not work properly in certain contexts like plugins because something has modified the path for some reason.

GUI apps not being able to access $PATH is a big one, I have found.

So it sounds like more of a packages thing than an interpreter thing; basically "DLL hell" all over again.

Setting aside all other axes of comparison, it sounds like Node really learned from Python with regards to this problem. Install Node itself, do an NPM install, and you're golden.

Until your global installs get stale/conflict with new global installs. npx is pretty handy for this problem though.

I would say the key is not relying on global installs. By default all NPM dependencies are installed locally alongside the manifest. You use more disk space, but disk space is cheap these days.

Clearly you have never found a random python script on the internet that does exactly what you need and tried to run it. It's a nightmare every single time.

Could be worse, try to run a C++ project next time! But yeah, the same applies to Ruby, Node, even C# (msbuild or CSX) and PowerShell modules. You have to know enough to know enough to get started.

About the only thing that reliably works besides "install Docker" is getting a bash shell going and pasting in a curl link, assuming your operating system is supported by whatever it is. That said, a close runner-up, that doesn't require bash and curl to be installed, would be Node's npx. Just run it, and "it works" even if whatever it is isn't installed. (Unless whatever it is doesn't have a precompiled binary for your system published and you then need to install CMake and compile tool chains like Xcode or Tools for VS)

Just a example for a breaking change between Python 3.7 and before: https://stackoverflow.com/q/51196568/6885801

It is because Python 3.7 makes "async" a reserved word, which was a valid identifier name before.

py34 asyncio was provisional (read: experimental, API in flux). It was very clearly labeled as such in both py34 and py35.[1][2] In py36 it was no longer provisional, but async/await syntax was established in py35 and asyncio.async was already deprecated back then. Using an example of a clearly experimental and evolving feature as breaking backwards-compatibility is disingenuous.

Now, there are actual backwards-incompatible changes that aren't made after at least two versions of deprecation, but those tend to be low level and subtle.

[1] https://docs.python.org/3.4/library/asyncio.html

[2] https://docs.python.org/3.5/library/asyncio.html

More on provisional API:

https://docs.python.org/3.8/glossary.html#term-provisional-a...

> A provisional API is one which has been deliberately excluded from the standard library’s backwards compatibility guarantees. While major changes to such interfaces are not expected, as long as they are marked provisional, backwards incompatible changes (up to and including removal of the interface) may occur if deemed necessary by core developers. Such changes will not be made gratuitously – they will occur only if serious fundamental flaws are uncovered that were missed prior to the inclusion of the API.

I think you misunderstood maple3142 comment: he is only saying that from Python 3.7 onward you could not use "async" as an identifier in your code, as it became a reserved word.

So maybe you were using "async" as a variable name, or as a function name in your code, and suddenly that wouldn't work anymore. It definitely is a backwards-incompatible change, and that has nothing to do with provisional APIs.

Since I have never use "async" as an identifier in my own code, I have no idea if warnings were issued, though probably?

Reserved words are introduced into languages over time. This is how languages evolve and hardly Python-specific. (Example: Rust added async/await keywords relatively recently.) You either stick with an old version or deal with it.

gp specifically links to a SO question discussing asyncio.async breakage, which would be considered API breaking except it was specifically provisional.

Even after the 2.x/3.x debacle, it's not as simple as keeping everything current. For example, until very recently, Tensorflow would work on Python 3.7 but not on 3.8. TF is just one well-known example, but version incompatibilities are common enough that many Python devs keep multiple virtual envs.