I’m pissed almost daily at how many build systems break because they need 3.5.3 vs 3.5.4 or whatever, and at how many only work because they create a new virtual environment for every single script (Virtual environment ~= copy of python auto-downloaded from some random places).

Also, FFS, why can’t pyc files have a bytecode format version number embedded in them, or at least be called .pyc3 by python 3?

The whole ecosystem needs to solve the packaging problem, or get replaced by some other language ASAP. Instead, it’s breaking entire distros with its packaging tire fire.

For many python projects I’ve dealt with, it is easier to reimplement that dang thing in some other language than make the python build reliable, secure and redistributable.

Doesn't Python 3 compile the .pyc files under the __pycache__ directory, and version them based on Python's version?

Maybe Python should stop changing so often?

With C++ for example you are usually either targeting C++11, C++14, or C++17 (or if you're really unfortunate, C++03). The set of features in the language only changes every few years.

Granted it's still possible that someone could have an old compiler that doesn't implement the advertised standard correctly, but that's a bug, not a deliberate design choice.

Thats not really fair. Python 3.2 was release in 2011 and Python 3.4 in 2014. Only 3.3 was released in the middle, I would not say it is only "so often" compared to c++. To be more general, C++ had 3 major versions from 2011 to 2017, while python had 5. Its more, but not significantly so (especially since very little compatibility is broken between 3.4 and 3.5 for example).

And if you are talking about which version is shipped with a distro, then RHEL7 ships with gcc 4.8, which support only parts of c++14, while for example the latest Ubuntu LTS ships GCC 7,3, meaning you get full C++17 support.

Don't get me wrong, the Python packaging situation is not very clean. But it is far from being an exception, and people complaining about it are just spreading FUD.

But the compiler is the same, so you specify the dialect you expect to use rather than running a different binary. Although to be fair, at least one distro decided to start shipping gcc like gcc-X where X is the version, so you can have multiple GCC's installed. I'm not sure that was the best idea, but its what they did.

> Maybe Python should stop changing so often?

This argument is almost cyclical.

> With C++ for example you are usually either targeting C++11, C++14, or C++17 (or if you're really unfortunate, C++03). The set of features in the language only changes every few years.

C++ went from glacial-pace changes - 85 / 98 / 03 / 11 -- to much faster ones: 14, 17, 20. This change in standards behavior was motivated by closing the gap with feature-rich, faster-moving languages like Python (Java, C#, etc).

Python makes language changes somewhat rarely, but library changes more often. The majority of the time, those library changes are brand new functionality.

#!/usr/bin/env python is almost guaranteed to give you a surprise at some point. If someone has a virtualenv or conda env active in that shell it will be a different python environment (with different installed modules) than in other shells.

At least in the build-system glue language use case, the 'batteries included' modules like os, sh, glob, etc will all be there.

Similarly, if you require pandas like I do, there's a fairly comprehensive baseline of functionality you can expect regardless of what environment it comes from.

Note: You shouldn't use #!/usr/bin/env python. Instead you should explicitly specify python2 or python3, unless your script is compatible with both.

You say "you" as if you're talking to the person you're replying to. Most scripts in the world aren't authored by the person you're replying to. They're powerless to change the ocean of python out in the world.

For example, out of about 50 or so /usr/bin/python scripts I see in a RHEL7's /usr/bin I can't find a single one that runs on python3.

No individual user is going to be fixing this. It's a very silly and shortsighted move on RedHat's part.

Of the 50 /usr/bin/ python files on my Ubuntu 16.04 desktop, most are versioned in some way. None are called from "env".

     37 /usr/bin/python3
      4 /usr/bin/python3.5
      4 /usr/bin/python
      3 /usr/bin/python2.7
      2 /usr/bin/python2

The 4 that aren't versioned are

speedtest, speedtest-cli, dh_python2, apt-offline

So two from ubuntu/debian, and two from a package

Right, someone working in ubuntu distro development has done this work for those packages specifically.

Above, we're discussing some build system where someone hasn't done this work. Most collections of python code in the world (I'm going to hazard a guess significantly upward of 90%) just use /usr/bin/python or /usr/bin/env python.

That's the nature of the problem.

You're discussing RHEL7's /usr/bin.

  > For example, out of about 50 or so /usr/bin/python scripts I see in a RHEL7's /usr/bin

I posted Ubuntu 1604's in comparison.

I'm discussing the problem of collections of python scripts using /usr/bin/python and I included the state of RHEL7 as an example.

The actual subject is CoolGuySteve's build system, which is likely to look like my example of RHEL7 rather than yours of ubuntu. The vast majority of environments look like RHEL7 or CoolGuySteve's build system where #!/usr/bin/python is the norm.

First, This is about RHEL 8 (Beta) not 7, 6, etc. Second `#!/usr/bin/python` is a different animal than `#!/usr/bin/env python`. One is hard coded. The other is `PATH` based so will change depending on current `PATH`.

On RHEL 7 (and earlier) `/usr/bin/python` must be the Python 2 the system shipped with or your break `yum` and other admin tools. When people try to install their own python by doing a `make install` as root, `yum` breaks and it's hard to recover the system.

What the article is saying is that RHEL 8 addresses this by having a platform python that the system tools use. So RHEL 8 tools will not use /usr/bin/python

CoolGuySteve complained that things will break because existing scripts assume /usr/bin/python. CoolGuySteve is correct. There is an enormous amount of work involved in changing or removing the function of /usr/bin/python.

Offering advice about how to write new software does nothing to help with the frustration he highlighted: His existing build systems will break. Build systems are especially frustrating as they tend to have hacked-together code that no one really wants to look at.

People will often work around this by installing a python2 as /usr/bin/python. At the end of the day this change results in a less predictable platform. On previous RHEL systems I know what /usr/bin/foo is -- it's predictable given the major version of the distro. But now I won't know what /usr/bin/python is on RHEL8. It will be uniquely different and that's not a good thing.

Totally agree in principle.

One interesting note - MacOS doesn't have a /usr/bin/python2 (at least, not at the time of this writing). It supplies /usr/bin/python, and also /usr/bin/python2.7.

That might not matter at all, but it's worth noting.

I know. I filed a bug about this in Apple's tracker years ago and they still haven't done anything about it. This contravenes the official upstream Python recommendation, which is that "Unix-like software distributions (including systems like Mac OS X and Cygwin) should install the python2 command into the default path whenever a version of the Python 2 interpreter is installed". See PEP-394 https://www.python.org/dev/peps/pep-0394/.

Which is fine but how do you specify dependencies right now? Surely a minimal install of RHEL doesn't have everything you need.

Not meant to be snarky, just a tad confused on what exactly you’re asking. Have you ever made an RPM yourself? Dependencies are specified there. Or are you asking about dependencies that don’t exist in standard RHEL repositories?

The parent was saying that specifying dependencies with rpm wasn't possible because they deploy with git but surely they must have something to specify/install deps since a RHEL base likely doesn't have everything they need especially if they're using the system python.

No doubt they have one of those "curl http://site.com/script.sh && sudo bash script.sh" installation instructions.

Sigh.

I think at that point we’re kind of stuck with virtual environments for Python dependencies. I’m not too sure what the best solution is.

Can't you just symlink your preferred python?

No need to symlink manually, the `alternatives` mechanism will handle that for you.