Developers should be using pyenv or similar (try Anaconda if you want "easy") and preferably one environment per project (so you can lock in library versions for the project).
I'd personally prefer the "OS python" to be more hidden, but I don't know enough about linux/mac inner workings to know if thats even a thing (maybe a libpython instead?).
But it is hard to understand if you aren't working with python all the time. I do basically just python, and still spent several years with outdated Ubuntu-shipped libraries before getting properly setup.
I really wish this was the default for all languages but the majority default to "install in your system" and then I run into the problems you and the OP mention.
As it is, finding out/figuring out how to work around the default is always magic incantations and spells buried on page 947 of appendix G if you're lucky or else spewed around the internet in drips and drabs by unreliable and or outdated sources.
As an example I wanted to contribute to some conformance tests for some browser API. The test suite used selenium which uses python all of which wanted global installs. I tried for a few hours to get it to run with local installs but I failed to find the correct secret ritual to perform and after a few hours decided I no longer felt like searching for the correct oracle to enlighten me.
ps: For my particular needs a VM was not a solution
Which is exactly what will happen later this year(?):
"The older Python language, version 2.7, is being deprecated in macOS 10.15 Catalina and won’t be included in macOS 10.16. The same goes for other UNIX scripting languages."
https://www.macobserver.com/analysis/macos-catalina-deprecat...
I think this is a mistake on Apple's part, fwiw.
It's really better for everyone if the base OS contains less, and you get more from the package managee (of which there are several to choose from on MacOS).
If Apple wants to provide installable tools on the OS install media, that's fine, of course.
So, no more "use the system python" for me...
I always use a language manager for Elixir, Erlang, Node, PHP, Python and Ruby, either the language's usual one or asdf. I'm even using different versions of databases, usually with docker. Java is somewhat easier to deal with, unzip in a directory and set JAVA_HOME but I'm not working much with it so I could miss some nuances.
Not for much longer.
"Scripting language runtimes such as Python, Ruby, and Perl are included in macOS for compatibility with legacy software. Future versions of macOS won’t include scripting language runtimes by default, and might require you to install additional packages. If your software depends on scripting languages, it’s recommended that you bundle the runtime within the app."
https://developer.apple.com/documentation/macos_release_note...
Is this the case? I thought Homebrew was considered fairly mediocre compared to more established package managers on (eg) Linux systems. But I've never been required to use a Mac, so my firsthand familiarity is limited.
Probably "wonderful" is subjective (Homebrew to me is far from being a wonderful piece of software), but "ubiquitous" is factually wrong. MacPorts, even though less popular than Homebrew, has a large userbase.
Quick tip:
pyenv which [python,pip,...]
pyenv help
I understand it’s another layer and for someone not working in it daily, it’s hard.
Some info, the command
which python
/usr/bin/python
python --version
pyenv version
If you want all versions installed,
pyenv versionsI can count on one hand the number of times “bundle install” has actually worked on the first try. Getting an existing project working on a new machine (sans docker) is an exercise in frustration.
Perhaps python should issue a warning if used outside a virtual environment, unless that warning is deliberately suppressed by a flag? That would be worth a point release on all older versions.
And I use it on Windows and Linux, server and desktop. You might be interested in: https://snarky.ca/what-the-heck-is-pyproject-toml/
Do python releases introduce backwards-incompatible changes (ignoring the 2.x/3.x divide)? Would it not be as simple has making sure you have the latest runtime?
In theory homebrew takes care of the whole thing for you but, in practice... as an only part-time OS X user I have run into a lot of problems with the path getting set in various ways that end up in various copies of the Python interpreter or library paths getting loaded, resulting in runtime errors due to library versions being inconsistent with the interpreter version or expected libraries not being loaded...
A further complication is that, the specific way that Apple vendored a set of python packages, it is possible but not particularly easy to add new packages to the OS-provided Python runtime, so most users, if they want some package that Apple didn't include, seem to end up installing a whole new environment just to get that package, even if the OS-provided environment were otherwise acceptable to them.
It's a very real problem and very frustrating, but I'm not quite willing to blame it on the Python ecosystem because it seems to be pretty much a result of Apple's decision to ship an interpreter and set of libraries that most users will end up being unhappy with. Anecdotally at least I run into these issues a lot less on Linux because even with fairly stable distributions (e.g. RHEL7.5/8) the included interpreter and library are more recent and the package repositories include most of the Python packages you would want, maintained to match the OS interpreter version. So while some people do end up using virtual environments it's less necessary than it is on OS X, where it frankly seems near impossible to do anything without installing a whole second Python environment and munging paths.
An extra complication is that, as a casual OS X user, it seems to me like OS X terminal emulators and tools are more likely to munge with the path than Linux terminal emulators and tools, so even if you get everything set up "just right" you end up having it not work properly in certain contexts like plugins because something has modified the path for some reason.
Edited to expand that, I guess what I mean to say but failed to articulate, is that the fundamental problem here (in my opinion) is that OS X ships with a Python distribution but not with a package manager. That's the fundamental difference from Linux where these problems are much less common. Third-party package managers like Homebrew and Macports are used, and in order to have full control of the Python environment they have to install a whole second copy because they are not allowed to tamper with the OS X Python environment. This pretty inevitably leads to frustration. The best solution would seem to be for Apple to provide some kind of "first-party-ish" package management functionality, perhaps something like Chocalatey which is not a Microsoft product but is built on Windows-provided primitives, so that the package managers that users are actually using can play nicely with the OS X Python environment.
This might be a function of the casualness rather than the tools. Here's a thing to check out on OS X:
man path_helper
Homebrew can help with distinction, letting the user install to `/usr/local/bin/python` while the OS continues to use `/usr/bin/python`. My personal solution is to install Python packages into my homedir (--user) or use virtualenvs. This also helps when six months from now you want a "clean" setup.
I've pushed any organization I've worked at who heavily relies on Python to decouple Python from the OS because of this and so OS updates can be independent of Python updates.
GUI apps not being able to access $PATH is a big one, I have found.
Setting aside all other axes of comparison, it sounds like Node really learned from Python with regards to this problem. Install Node itself, do an NPM install, and you're golden.
About the only thing that reliably works besides "install Docker" is getting a bash shell going and pasting in a curl link, assuming your operating system is supported by whatever it is. That said, a close runner-up, that doesn't require bash and curl to be installed, would be Node's npx. Just run it, and "it works" even if whatever it is isn't installed. (Unless whatever it is doesn't have a precompiled binary for your system published and you then need to install CMake and compile tool chains like Xcode or Tools for VS)
It is because Python 3.7 makes "async" a reserved word, which was a valid identifier name before.
Now, there are actual backwards-incompatible changes that aren't made after at least two versions of deprecation, but those tend to be low level and subtle.
[1] https://docs.python.org/3.4/library/asyncio.html
[2] https://docs.python.org/3.5/library/asyncio.html
More on provisional API:
https://docs.python.org/3.8/glossary.html#term-provisional-a...
> A provisional API is one which has been deliberately excluded from the standard library’s backwards compatibility guarantees. While major changes to such interfaces are not expected, as long as they are marked provisional, backwards incompatible changes (up to and including removal of the interface) may occur if deemed necessary by core developers. Such changes will not be made gratuitously – they will occur only if serious fundamental flaws are uncovered that were missed prior to the inclusion of the API.
So maybe you were using "async" as a variable name, or as a function name in your code, and suddenly that wouldn't work anymore. It definitely is a backwards-incompatible change, and that has nothing to do with provisional APIs.
Since I have never use "async" as an identifier in my own code, I have no idea if warnings were issued, though probably?
gp specifically links to a SO question discussing asyncio.async breakage, which would be considered API breaking except it was specifically provisional.
pythonX.Y -m moduleThis approach doesn't help--it makes things worse. I see this occasionally on Linux as well. A novice programmer will think: My program depends on programs X, Y, and Z, and if they're not there, I'll download them from the Internet to make things easier for my user.
Aside from being a major security hole, this just makes a mess. The user has no idea of what's going on, and it's 10x harder to debug when something goes wrong.
I do relate to your pain. For starters, I'd recommend not using a Mac for Python development (or really any development). If you must, something like a VirtualBox machine running Linux would be good. But probably better to just make a Linux instance in the cloud.
I develop on a Mac where in theory everything should be easier:
* Comes with Python out of the box
* Platform has ubiquitous and wonderful package manager (Homewbrew)
And yet, 90% of my interactions with Python via the CLI involve it crashing because the version that the current thing I'm trying to compile or execute wants isn't the one the OS happens to be vending for it.
Some OSS projects suggest using things like pyenv to help. I don't know, it just seems to make things even worse for me because now even 'which python' doesn't tell me anything because it directs to some shell script shims that I can't parse or understand.
Python the language is wonderful, as are the libraries that exist for it.
Python the runtime/execution environment is actively hostile to users, and it's honestly keeping me from actually building anything real with it from the ground up.
I'm not surprised that Microsoft went with this approach to TRY to help it's users.
I'm also not surprised that Python devs are pissed because I've also been in those shoes where FINALLY everything is working, and you think you have that one version of python 2 and one version of python 3 both running and coexisting and everything you need to run runs, and then something small changes and WHAM the whole house of cards collapses.
But I wouldn't be angry at Microsoft, I'd be angry at the Python community for letting things get so bad.