What's amazing is that pip3 gladly downloads and tries to install py2 stuff. Because package/dependency management is a complete afterthought in Python.

And sure, it'll get better, and finally we have lockfiles (Pipenv), and maybe eventually a proper SAT solver will help resolving dependencies https://github.com/pypa/pip/issues/988#issuecomment-36084645... , aaand gradual typing is nice too, and maybe eventually we'll have a proper async library (python-trio, as asyncio is there, but not low level and/or not ergonomic enough).

Yet Python is free and open, and amazing, and I haven't contributed much other than report bugs, so I'm not complaining, just comparing to other ecosystems.

Conda has a silver and has been available for however many years. It also works well with pip.

Among people complain about python package management, I have never heard an argument against Conda other than “I don’t like that they recompile base python”. What’s your reason to discount conda?

Our python is compiled with the latest versions of gcc (on Linux) and clang (on macOS). So you're getting gcc 7.2 compiled python on RHEL/CentOS 6. And all of the security and performance enhancements that come along with it [1]. (Ok we're still on 7.2; we'll update to 7.3 in the next couple months.)

[1] https://www.anaconda.com/blog/developer-blog/improved-securi...

P.S. If you're looking for a latest and greatest C/C++ compiler toolchain that's backed with "production" testing via hundreds of millions of package installs and works on older versions of linux...

It's time!

It's really irritating in the scientific programming world at the moment because we've had some dependencies take years to get to Python 3 compatibility and are they're now being all holier than thou about it all.

We couldn't even start doing the work until our biggest dependency (which we use in 90% of the code base) was ported to Python 3 and while that's been available for about a year, they weren't distributing it and you had to compile from source to get the Python 3 version - and compiling this particular package, which shall remain nameless, is one of the most difficult compilations I've ever had the misfortune to try. It's notable by it's absence on the UK national supercomputer because of this, for example.

So now, we're trying to port our code to it, but it's likely going to take 6 months to a year to do so. Our day job is doing scientific research, and that's what brings in grant income - so for us, it's not a huge priority to spend all our time working on the switch to Python 3, because we can run everything in a Docker container with fixed dependencies and it should continue to run fine for the next 10 years. It will happen, it's just not the most important thing on our list.

PySpark?

It’s not that surprising. People want to work on their tasks of interest. Smart programmers allocate some time for maintenance activity but py3 is a rewrite that is well outside the boundaries of normal maintenance. When the first thing you run has syntax errors and you might have to change almost every file to make any progress, it’s a rewrite.

And plus, many projects are maintained by people in their free time, often for free. Of course no one wants to spend a ton of free time redoing everything only to work apparently the same as before.