News for hackers. Stuff that matters.
Re:Arch Linux pulls the plug on 32-bit by CJefferson on Thursday February 09, 2017 @09:37AM (#13607167)
The switch to the 'python' command running python 3, as a non-arch user, put me off Arch forever. It just broke everything, which had long assumed 'python' would run python 2.

Not installing python 2, and just python 3, with the name python3, would have been fine.

Arch put us in a situation where it was basically impossible to run python 2 with a #! line, as some distros hadn't yet introduced a 'python2' symlink yet.

Re:Arch Linux pulls the plug on 32-bit by StreakyCobra on Thursday February 09, 2017 @10:17AM (#13607472)
So to avoid re-opening the debate of Python 2 vs Python 3, let's take this point of view: The Python 3 language may, or may not, have been an improvement, and the transition from 2 to 3 may, or may not, have been done correctly. Everyone has its own opinion on this. But at the end this problem concerns the Python community.

From the Arch developers point of view, the Python developers decided to release a new version of their language. They were going to support python 2 for a while, but the direction that the language was taking for the future was Python 3, so one day or another Python 3 would become the standard, so doing the switch was the good decision.

As said by @tbranyen the work of adapting shebangs have been done on the Arch side anyway, so I don't see a problem for people developing their projects in Python. Also Arch users are mostly power users (you need to understand quite a bit about linux to install it) so they are usually able to handle python 2 vs 3 errors quite well, especially as it is known than Arch use Python 3 by default.

Re:Arch Linux pulls the plug on 32-bit by bbatha on Thursday February 09, 2017 @11:00AM (#13607813)
Except that python says that python 3 should be an executable named `python3` and `python` should refer to `python2` [0]

0: https://www.python.org/dev/peps/pep-0394/

Re:Arch Linux pulls the plug on 32-bit by jvreeland on Thursday February 09, 2017 @11:58AM (#13608450)
It says right in the pep

"...however, end users should be aware that python refers to python3 on at least Arch Linux (that change is what prompted the creation of this PEP), so python should be used in the shebang line only for scripts that are source compatible with both Python 2 and 3..."

There was no guidance before the Arch devs made their choice. There's no reason to change back.

Especially since the goal is that python3 will be default eventually.

"...in preparation for an eventual change in the default version of Python, Python 2 only scripts should either be updated to be source compatible with Python 3 or else to use python2 in the shebang line."

Re:Arch Linux pulls the plug on 32-bit by srssays on Thursday February 09, 2017 @01:40PM (#13609574)
There's a good reason to change back. There is now convention in place, and Arch's approach is obsolete.
Re:Arch Linux pulls the plug on 32-bit by lelandbatey on Thursday February 09, 2017 @02:28PM (#13610032)
It seems to me that Arch established their convention first, so why should Arch "change back" when they where first; why are others not "changing back" to Arch's convention, given that Arch's came first?

Well, the answer is that this is a silly discussion. Arch is doing what Arch is doing because Arch is intentionally forward oriented, sometimes to the detriment of backwards compatibility with the old school (and yes, a distro not having a symlink for python2 is at this point nearly 5 years into being "old school").

Re:Arch Linux pulls the plug on 32-bit by srssays on Thursday February 09, 2017 @03:29PM (#13610625)
A particular convention has been adopted by nearly all distros, the Python project and the Python community. That convention is better than Arch's, because it is backwards-compatible with the large body of Python 2 code that exists.

Arch's change angered many in the Python community, and broke a lot of code. That Arch refuse to adopt the established convention, because it would involve admitting a mistake, shows a distinct lack of professionalism.

Re:Arch Linux pulls the plug on 32-bit by StreakyCobra on Thursday February 09, 2017 @04:08PM (#13610999)
I don't understand why people always want to tell other people how to behave. The change was decided by Arch developers and impacted only the Arch distribution. Arch developers and Arch users did their job by adapting the Arch packages to work with this. I don't see any problem. If users don't like this choice, they are free to use any other distribution. If software developer don't like this choice, they are also free to not support Arch. And even this is not a problem most of the time because Arch users are doing the effort of adapting and packaging applications themselves on AUR.
Re:Arch Linux pulls the plug on 32-bit by khedoros1 on Thursday February 09, 2017 @06:32PM (#13612045)
> The change was decided by Arch developers and impacted only the Arch distribution.

Well, until developers that do their work on other distributions start getting bug reports as a result of Arch's unexpected naming convention. Should someone outside of the distribution expect Arch to change so that their software works there? No. Should Arch expect outside developers to adapt their code to work on Arch? No. Honestly, I'd expect the kind of people that Arch attracts to be used to dealing with little snags in operation, and for them to just dive in and fix it on their own.

Re:Arch Linux pulls the plug on 32-bit by StreakyCobra on Thursday February 09, 2017 @11:11AM (#13607899)
Fair point. But Archlinux made the switch in 2010 [1], and this PEP was created in 2011.

[1] https://www.archlinux.org/news/python-is-now-python-3/

Re:Arch Linux pulls the plug on 32-bit by blipmusic on Thursday February 09, 2017 @01:59PM (#13609750)
Using the official, graphical installer for Python 3 on Windows with no previous version installed creates an executable 'python' that refers to Python 3.

(Normally not a Windows user so I have no idea why this is the case or if by design.)

Re:Arch Linux pulls the plug on 32-bit by groovy2shoes on Thursday February 09, 2017 @02:20PM (#13609951)
Windows doesn't support shebang lines, so the exact name/location of the interpreter doesn't matter as much as it does on 'nix.
Re:Arch Linux pulls the plug on 32-bit by maxerickson on Thursday February 09, 2017 @03:47PM (#13610803)
Recent versions of python ship with a launcher shim for windows that does support the shebang line (to simplify using multiple interpreters):

https://www.python.org/dev/peps/pep-0397/

As you say though, it doesn't rely on the names of the exes.

Re:Arch Linux pulls the plug on 32-bit by CJefferson on Thursday February 09, 2017 @10:31AM (#13607570)
I work on two programs which either not distributed by Arch, or our users choose to download directly (I don't know).

They both contained Python scripts, and I had to waste my time figuring out why they had broken when I had e-mails from Arch users complaining they didn't work -- and then keep replying to future Arch users with the fix.

I don't know how many hours and days of other developer's time was wasted by Arch with this change, but it certainly wasted a good day of my time over the last few years.

Re:Arch Linux pulls the plug on 32-bit by Latty on Thursday February 09, 2017 @11:10AM (#13607895)
I'm an Arch user, and it literally broke nothing for me. Packages were updated, life went on.

You seem to be complaining because other distros hadn't done the right thing. I don't see how that's Arch's fault. Are we to hold back for the lowest common denominator? Do we need every distro to join together and agree to a switchover date?

There was official guidance from Python on the switchover, this wasn't some maverick decision. It was just moving forward.

Not everyone wants to be tied to ancient stuff for backwards compatibility - if you do, it's your job to deal with that.

Re:Arch Linux pulls the plug on 32-bit by rlpb on Thursday February 09, 2017 @12:28PM (#13608796)
My Python 2 scripts on Github start with "#!/usr/bin/python", as do many others. All of these broke for you as an Arch user.

Because Arch did what they did, Python now recommends that Python 2 scripts start "#!/usr/bin/python2" (or the env equivalent).

> Are we to hold back for the lowest common denominator? Do we need every distro to join together and agree to a switchover date?

It's not just other distros; it's the rest of the world, including all of the scripts that people run but distros don't necessarily ship.

The right way to migrate would be to ship and use both /usr/bin/python2 and /usr/bin/python3. Leave /usr/bin/python as a symlink to python2 for a while. Eventually, drop the symlink, but do not replace it with python3. Allow users to opt-in to a compat symlink if they wish. Let stuff catch up. When the expectation that "python" is python2 has faded, then introduce a symlink from python to python3 by default, letting users opt-in earlier if they wish.

> There was official guidance from Python on the switchover, this wasn't some maverick decision.

No, there wasn't, and yes, it was some maverick decision. It was done without consultation with upstream.

(I am a non-Arch distribution developer)

Re:Arch Linux pulls the plug on 32-bit by Latty on Thursday February 09, 2017 @02:49PM (#13610239)
This just feels like culture clash to me - the whole change to Python 3 was about accepting the cost of breaking changes to avoid dragging around useless cruft and bad decisions forever.

The idea that we should then never use `python` to mean `python3` is just so backwards. Sure, in your corporate environment where backwards compatability is suepr important, do that.

In Arch, people have a distro that moves fast and breaks things, and we are fine with that. I'll accept the cost of occasionally having to change a shebang line (although as I have almost never had to go outside the AUR - someone else in the Arch community has almost always dealt with this for me).

> No, there wasn't, and yes, it was some maverick decision. It was done without consultation with upstream.

Fair enough, I got my timelines mixed up - but to be fair, it was then ratified as the correct way to do things because the upstream project agreed with the core idea.

> The right way to migrate would be to ship and use both /usr/bin/python2 and /usr/bin/python3. Leave /usr/bin/python as a symlink to python2 for a while. Eventually, drop the symlink, but do not replace it with python3. Allow users to opt-in to a compat symlink if they wish. Let stuff catch up. When the expectation that "python" is python2 has faded, then introduce a symlink from python to python3 by default, letting users opt-in earlier if they wish.

I don't really see why I should have to do this manually? If you don't want that behaviour, use another distro. This just feels like you feel like you should have some say in how other people set up their systems.

If you really hate it so much, don't support Arch - that'd be fair. Trying to shame the distro for doing it ignores the fact it's the core idea of the distro to do stuff like that, that's the point.

Re:Arch Linux pulls the plug on 32-bit by rlpb on Thursday February 09, 2017 @03:06PM (#13610413)
> The idea that we should then never use `python` to mean `python3` is just so backwards.

I did not present any such idea. I presented a sane migration path to where `python` means `python3`.

> ...it was then ratified as the correct way to do things because the upstream project agreed with the core idea.

No. It was because the upstream project had their hand forced and are pragmatic about these things.

> I don't really see why I should have to do this manually?

In my proposal for a sane migration path to the ideal? You wouldn't have to do it manually. The distribution would do it. As a user you'd be able to override it to speed up the migration for yourself if you chose; that's all.

Re:Arch Linux pulls the plug on 32-bit by Latty on Friday February 10, 2017 @06:48PM (#13619950)
I chose to run a bleeding edge distro to be on the bleeding edge. Requiring me to make the changes to be on the bleeding edge manually makes no sense.

Expecting that is as weird as expecting super stable LTS releases to run the latest-and-greatest of everything.

Re:Arch Linux pulls the plug on 32-bit by rlpb on Saturday February 11, 2017 @06:18AM (#13622151)
> Requiring me to make the changes to be on the bleeding edge manually makes no sense.

Which part of "You wouldn't have to do it manually" do you not understand?

Re:Arch Linux pulls the plug on 32-bit by Latty on Saturday February 11, 2017 @11:30PM (#13626741)
Which part of "I want it now, not in the future" do you not understand?

> In my proposal for a sane migration path to the ideal? You wouldn't have to do it manually. The distribution would do it. As a user you'd be able to override it to speed up the migration for yourself if you chose; that's all.

I want to speed up the migration for everything - that's why I run Arch. That 'override' is a manual step I don't want to have to do.

Re:Arch Linux pulls the plug on 32-bit by AsyncAwait on Thursday February 09, 2017 @12:42PM (#13608961)
> My Python 2 scripts on Github start with "#!/usr/bin/python", as do many others. All of these broke for you as an Arch user.

So you made certain assumptions. Have you made your scripts use python2, which is safer anyway, everything would've been fine, don't blame Arch for your incorrect assumptions.

Re:Arch Linux pulls the plug on 32-bit by rlpb on Thursday February 09, 2017 @12:47PM (#13609000)
No. Before Arch made their change, that /usr/bin/python is Python 2 was the only possible assumption that could be made. No distribution shipped /usr/bin/python2 at that time. There was no need, because /usr/bin/python was always Python 2.

It was only because of what Arch unilaterally did that forced the community to start providing a /usr/bin/python2. Before that, /usr/bin/python2 did not commonly exist at all.

Re:Arch Linux pulls the plug on 32-bit by AsyncAwait on Thursday February 09, 2017 @01:14PM (#13609296)
> It was only because of what Arch unilaterally did

Arch is a bleeding-edge, latest software kind of distribution. They did not made the decision 'unilaterally'. They made the decision to ship the latest software for their own distribution, as they always do. It may have proven that a lot of software is extremely poor/inflexible, (i.e. rests on weak assumptions), but I am glad Arch moved forward as that showed their resolve to move technology forward even in the face of a lot of pressure from outside groups.

I just think that instead of blaming Arch, try to make your scripts more robust, i.e. loop on all 'python*' binaries in /usr/bin and use the first one whose --version gives you '2.x' instead of relying on /usr/bin/python being python 2 when the latest version is 3 and Arch is known to ship latest software.

Re:Arch Linux pulls the plug on 32-bit by rlpb on Thursday February 09, 2017 @03:02PM (#13610373)
> They made the decision to ship the latest software for their own distribution, as they always do.

As did all the other distributions. Shipping Python 3 is not the issue here. Breaking compatibility with all existing Python 2 scripts is what they did. The two are not mutually exclusive.

> I just think that instead of blaming Arch, try to make your scripts more robust, i.e. loop on all 'python<glob>' binaries in /usr/bin...

You want me to do that in a shebang line? No. That'd be crazy.

> ...when the latest version is 3 and Arch is known to ship latest software.

See https://en.wikipedia.org/wiki/Application_binary_interface. What Arch did was break the ABI of what /usr/bin/python means, breaking all scripts that relied on that ABI. Shipping the latest software is orthogonal to this. The rest of the world ships the latest software too, but without breaking ABI.

Re:Arch Linux pulls the plug on 32-bit by AsyncAwait on Friday February 10, 2017 @04:37AM (#13614291)
The rest of the world explicitly doesn't ship the latest software, check the kernel version in the latest version of ubuntu vs arch, same for GNOME etc. Ubuntu did not even had python3 installed for a long time by default, you had to apt-get it

The ABI was never documented and at best a convention-enforced one.

Re:Arch Linux pulls the plug on 32-bit by rlpb on Friday February 10, 2017 @07:26AM (#13614893)
> Ubuntu did not even had python3 installed for a long time by default, you had to apt-get it

I call your straw man. As I've already repeatedly said, the issue is about breaking ABI, not about what software distributions shipped, nor what they had installed by default. Making /usr/bin/python point to Python 3 has nothing to do with what distributions shipped, nor what they had installed by default. The issues are completely orthogonal. Stop trying to pretend otherwise.

> The ABI was never documented and at best a convention-enforced one.

Upstream shipped build systems that put Python 2 in /usr/bin/python, and Python 3 in /usr/bin/python3. That's about as good a definition of ABI as one gets in the free software world. Arch deliberately patched the ABI. You can try to argue that their decision to do so was correct, even though I disagree. You cannot argue that they didn't know they were changing an ABI when they patched the ABI.

Re:Arch Linux pulls the plug on 32-bit by AsyncAwait on Friday February 10, 2017 @08:12AM (#13615118)
> call your straw man. As I've already repeatedly said, the issue is about breaking ABI, not about what software distributions shipped,

But it is very much about that. Arch ships latest software, Python 3 is the latest python, python 2 is not. Ubuntu, Debian etc. ship old, outdated software regularly, Arch doesn't.

Because the latest python was python 3, /usr/bin/python pointed there. The other distributions explicitly decided not to ship the latest python.

Prior to Arch making the switch, the 'ABI' was not /usr/bin/python points to python 2, it simply pointed to the latest python release, which on most systems happened to be python 2, because they did not even ship python 3 at all at the time.

You may not agree with the decision, but to pretend like it was "out of nowhere" is unhelpful.

Re:Arch Linux pulls the plug on 32-bit by RUG3Y on Thursday February 09, 2017 @02:18PM (#13609930)
The decision is pretty simple: if a user doesn't like the way Arch does things, they're free to use any of a hundred other distros. I'm with you, I think this is fine.
Re:Arch Linux pulls the plug on 32-bit by simplyluke on Thursday February 09, 2017 @12:55PM (#13609085)
> Are we to hold back for the lowest common denominator?

Well, the fact that over a decade after the release of python 3 we're _still_ arguing about it shows that lot of people think we should.

Re:Arch Linux pulls the plug on 32-bit by CJefferson on Thursday February 09, 2017 @11:13AM (#13607919)
I thought the official advice was that 'python' should remain 'python2'?

from https://www.python.org/dev/peps/pep-0394/ (not yet replaced/update) "for the time being, all distributions should ensure that python refers to the same target as python2 ."

Re:Arch Linux pulls the plug on 32-bit by StreakyCobra on Thursday February 09, 2017 @11:19AM (#13607988)
Archlinux made the switch in 2010 [1], and this PEP was created in 2011. [1] https://www.archlinux.org/news/python-is-now-python-3/
Re:Arch Linux pulls the plug on 32-bit by michrassena on Thursday February 09, 2017 @10:31AM (#13607572)
What I wonder is why one would assume the python command would run python 2, instead of running python 1, assuming python 1 was a thing. By your logic, shouldn't python2, run python3 run python 3, etc. What I'm getting at, is we get used to things being a certain way, but often we're just following a convention that was set in place before we got onboard. Why do we still type "bash" instead of "bash4"?
Re:Arch Linux pulls the plug on 32-bit by CJefferson on Thursday February 09, 2017 @10:34AM (#13607591)
The reason you would assume it is that, for a long time on every Linux distribution, there was a single way to run python2, and it was called 'python'. I'm not sure what you are getting at, this was just the way it was. You couldn't run python 2 any other way in some linux distros.

The reason we don't need a 'bash4' is that, by and large, bash tried very hard to make sure they didn't break any bash3 scripts, so there is no need for old bash3 scripts to know they are now running in bash4, unlike python3.

Re:Arch Linux pulls the plug on 32-bit by michrassena on Thursday February 09, 2017 @10:54AM (#13607753)
I get the point, python 2 scripts aren't fully compatible with python 3. I've written a little python and know that to be the case.

However, I can predict with near certaintly, if python 4 doesn't break python 3 scripts, people will complain if "python3" doesn't invoke python 4, because they've always used "python3" to run the new python and "python" to run python 2.

Re:Arch Linux pulls the plug on 32-bit by gravypod on Thursday February 09, 2017 @02:29PM (#13610044)
> Why do we still type "bash" instead of "bash4"

Because Bash was made by, and is maintained by, people who understand how backwards compatability in their program is important.

Re:Arch Linux pulls the plug on 32-bit by hobarrera on Thursday February 09, 2017 @09:05PM (#13612745)
Just like python. That's why they bumped the major when release backwards-incompatible changes, which is the right thing to do.
Re:Arch Linux pulls the plug on 32-bit by AsyncAwait on Thursday February 09, 2017 @12:39PM (#13608923)
Why not be explicit with 'python2' and be done with it?
Re:Arch Linux pulls the plug on 32-bit by Zancarius on Thursday February 09, 2017 @01:23PM (#13609400)
I agree, but I'd guess the chief complaint is that doing so requires the `python2` symlink to be available. There's at least one claim up thread that it doesn't exist on all systems (create it yourself?). I didn't catch the distribution used, but I DID check an older Ubuntu image I have (I run Arch) and there's definitely a python2 link pointing to python2.7 (`python` also points to python2.7 and there's a `python3` symlink that does what you'd expect). So outside some LTS build, it seems surprising to me that it wouldn't exist in 2017 on a modern distro. I still can't shake the feeling that the thread started around a relatively insignificant complaint (dumping Arch because they use Python3 by default? Come on...).

Personally, I'd rather just use `/usr/bin/env python2` to invoke the appropriate interpreter (if you don't need to pass arguments to it), and if the symlink doesn't exist, the onus is on the user to create it. This is ESPECIALLY true for older distros.

Plus, for a sufficiently complicated application, why would someone want to pollute the global site-packages anyway? I'd rather manage most things inside a virtualenv or similar (pick your Python version), because I've seen far too many install_requires/requirements locking to a specific library version.

Re:Arch Linux pulls the plug on 32-bit by CJefferson on Thursday February 09, 2017 @01:32PM (#13609491)
In 2017, indeed everyone has added python2, but I assure you it wasn't that way when Arch originally did the python3 -> python rename.

While this is a small issue, this original thread used it as an example of how Arch is good and pragmatic. To me it seemed like the opposite -- it broke lots of code and packages for no good reason. Why should users of older distros have to add a symlink, when before Arch everyone could be sure that '/usr/bin/python' if present would be python2, and '/usr/bin/python3', if present, would be python3?

Now I agree that using virtualenv or similar is a good idea, but this actually caused the most problems for little 20 line python scripts, because often the users of those didn't even realise they were using python.

Re:Arch Linux pulls the plug on 32-bit by Zancarius on Thursday February 09, 2017 @02:06PM (#13609828)
> I assure you it wasn't that way when Arch originally did the python3 -> python rename.

I disagree. I don't believe it was as painful as some make it out to be. The official packages were updated within a relatively short period of time, and while the AUR packages took longer they're also not officially supported. There was also a news announcement [1]. Honestly, I think of a few other transitions over my years of using Arch as being far, far more painful. The Python2 -> Python3 defaults change isn't one of them.

Although I would agree that Gentoo's approach was somewhat better via eselect, which I believe predated Arch's migration.

> Why should users of older distros have to add a symlink

You're right. That should be the responsibility of the package maintainer. Conversely, why should I, as a developer, have to continue assuming `/usr/bin/python` points to any particular version of Python when most distributions have migrated away from this anachronism? sed does magic, and you really should be using the appropriate symlink for your desired version anyway. Python 3 isn't new. I have a hard time seeing this as problematic because 1) things change and 2) the solution is easy. I'd be happy to entertain a use case where this actually has presented material difficulties, however.

> but this actually caused the most problems for little 20 line python scripts, because often the users of those didn't even realise they were using python.

Well, I do agree: It's a problem for the end user, but the solutions aren't difficult (if someone's using Linux, they probably know enough to fix it). So, either change the hashbang, add a symlink, and--failing that--complain to the package maintainer if it's been packaged up by the distro and the upstream software itself has changed (it should).

My recommendation for developers would be to use the appropriate /usr/bin/python* symlink. Nearly all distros have them in place now (including Ubuntu from a couple years ago). Perhaps assuming /usr/bin/python will always point to a fixed version is a matter of misplaced assumptions.

[1] https://www.archlinux.org/news/python-is-now-python-3/

Re:Arch Linux pulls the plug on 32-bit by AsyncAwait on Thursday February 09, 2017 @01:49PM (#13609647)
> for no good reason.

There was good reason, to move to the latest version of a particular software, since that's kind of the whole philosophy of Arch, in fact the reason I originally switched to it in 2011 was because I had enough of me not being able to use the latest software on Ubuntu.

> Why should users of older distros have to add a symlink, when before Arch everyone could be sure that '/usr/bin/python' if present would be python2

If you made no effort to detect the actual version, it's wrong to assume that. It could've just as well been python 1.

Re:Arch Linux pulls the plug on 32-bit by Zancarius on Thursday February 09, 2017 @03:20PM (#13610527)
> in fact the reason I originally switched to it in 2011 was because I had enough of me not being able to use the latest software on Ubuntu.

It's funny. My reason for switching to Arch was almost the opposite. I grew tired of having to rebuild the latest software in Gentoo every time I updated and ran into someone on Slashdot who encouraged me to give Arch a try. Sure, there's more prebuilt packages, mirrors, etc for Gentoo these days, but then you're in the same boat as everyone else (waiting for the package to be built).

Re:Arch Linux pulls the plug on 32-bit by RUG3Y on Thursday February 09, 2017 @02:21PM (#13609967)
You're right. - If Arch is targeted for power users, what's preventing them from configuring symlinks etc? - Why aren't they using virtualenvs? I never install Python packages outside of a virtualenv.
Re:Arch Linux pulls the plug on 32-bit by Zancarius on Thursday February 09, 2017 @03:28PM (#13610612)
> I never install Python packages outside of a virtualenv.

Global site-packages is all fun and games until someone loses an eye (or a dependency).

I had a fun learning experience with a large Python package once while trying to mix and match dependency versions, simultaneously trying to avoid clobbering the ones already installed by pacman. Without a virtualenv, it quickly becomes impossible to guarantee you won't break something in the process of installing something else. I never made that mistake again.

Re:Arch Linux pulls the plug on 32-bit by hobarrera on Thursday February 09, 2017 @09:07PM (#13612751)
You really shouldn't install global packages circumventing the package manager (this applies to python and everything else really). You're just asking for eventual breakage, or leaking obsolete files.
Re:Arch Linux pulls the plug on 32-bit by Zancarius on Thursday February 09, 2017 @09:58PM (#13612921)
Exactly. In retrospect, I'm actually not sure I've done something quite so stupid in a long time. However, I'm sure I did it ~10 years ago on Gentoo and/or FreeBSD when I was much stupider about dependencies. Either way, what I should've said is outlined below in my last paragraph, and mostly what I mean by polluting global site-packages (with PKGBUILDs!)--and trying not to clobber things.

As an aside first: I sometimes make a PKGBUILD for most things just on the merit that it's better for pacman to manage site-packages on its own--or anything, really. It's a terrible idea to stuff things into global library directories that aren't managed in general (although incautious use of things like npm or pip can certainly do that for you--still an awful idea).

However, I do recommend installing packages into a virtualenv for anything with modest complexity, because some applications have requirements/install_requires that lock specific versions which may not be mutually compatible with others, and there's also the circumstance where official packages aren't new enough but you still want the package manager to maintain the dependent package. I guess you could go with versioned packages in this case, but since the official ones will inevitably find themselves updated at some point, I see it as easier to circumvent the issue in the first place via a virtualenv.

Anyway, I did the above polluting nonsense once when building a package with quite a few dependencies: Trying to maintain a half dozen individual PKGBUILDs just for dependencies and pestering easily a half dozen other maintainers to update their respective PKGBUILDs versus modifying the application's setup.py (either with patch or sed) to match older versions were both terrible ideas and scaled poorly. It's far easier to simply build what's needed in a virtualenv and avoid polluting the global site-packages even with the package manager handling them on its own. This is especially true for complex applications with many dependencies that may already be installed (but with conflicting versions).

Sorry for the rant, you just reminded me and tickled all the right spots! :)

Re:Arch Linux pulls the plug on 32-bit by tbranyen on Thursday February 09, 2017 @09:43AM (#13607208)
It really wasn't that bad and the AUR + official channels had patched packages with the correct shebang line. The worst offender for a while was the node-waf builder, had to patch it manually to get any packages to build, otherwise you needed a command line arg every time. This was like 5 years ago tho... These days I have never hit an issue w/ a properly installed python package.
Re:Arch Linux pulls the plug on 32-bit by throwaway7767 on Thursday February 09, 2017 @10:41AM (#13607643)
Those scripts should probably be using something like the following, if they really are python2-only: #!/usr/bin/env python2

I get your frustration though, it's a change that breaks a whole lot of existing scripts.

Re:Arch Linux pulls the plug on 32-bit by maxerickson on Thursday February 09, 2017 @10:58AM (#13607793)
That doesn't address the part where python2 didn't exist in a lot of places at the time, so scripts using python2 to work on Arch would have to be adjusted to work elsewhere. There was no universal solution.
Re:Arch Linux pulls the plug on 32-bit by hobarrera on Thursday February 09, 2017 @09:00PM (#13612727)
The thing is, python guidelines (PEPs) stated that shebangs should specify python2 or python3 explicitly during the transition period. Using "#!/usr/bin/env python" for a project that was python2-only was not following guidelines, so the breakage was really the dev's fault.

Also, the python2 symlink should have already been there. Bad packaging on some distros resulted in bad practices packaging by some devs. Don't blame that on Arch.

Re:Arch Linux pulls the plug on 32-bit by rlpb on Friday February 10, 2017 @08:18AM (#13615141)
You're getting the ordering of events wrong. Arch's actions weren't in response to the PEP. The PEP was written in response to Arch's actions.

> Also, the python2 symlink should have already been there.

Nope. There was no such convention or specification at the time. There was, however, convention and specification (in the form of "what Python upstream build systems do") on what /usr/bin/python means, which is "Python 2".

Re:Arch Linux pulls the plug on 32-bit by hobarrera on Sunday February 12, 2017 @04:11AM (#13627529)
> There was no such convention or specification at the time.

There was no formal specification, but there was a convention that many distributions were already following (and quite a few others were not).

Re:Arch Linux pulls the plug on 32-bit by Fiahil on Thursday February 09, 2017 @09:56AM (#13607319)
Well, even Ansible folks started to switch to python 3 (https://docs.ansible.com/ansible/python_3_support.html). Pushing people off python 2 is not "absolutely evil".
Re:Arch Linux pulls the plug on 32-bit by CJefferson on Thursday February 09, 2017 @11:14AM (#13607938)
I have no problem with Arch having a "only install python3 by default" rule. They can even not distribute python2 for all I care.

But, don't make /usr/bin/python run python3, that just confuses programs which have made the (reasonable based on past experience, and official python advice) assumption that it /usr/bin/python will run python2.

Re:Arch Linux pulls the plug on 32-bit by simplyluke on Thursday February 09, 2017 @12:57PM (#13609116)
So forever into the future $ python should run python2? That's not pragmatic at all.
Re:Arch Linux pulls the plug on 32-bit by marvy on Thursday February 09, 2017 @07:46PM (#13612423)
On Ubuntu, "sqlite" starts sqlite2.x; if you want sqlite3.x, you must type "sqlite3". The last version of sqlite2 came out in 2005. As far as I know, hardly anyone still uses sqlite2 for anything anymore. (I could be very wrong though.) Nevertheless, people must still type that extra character "3". The sky has not yet fallen :)

It seems to me that the same approach could work for Python.

Re:Arch Linux pulls the plug on 32-bit by michaelmrose on Thursday February 09, 2017 @02:25PM (#13610005)
Why not? Can you elaborate?
Re:Arch Linux pulls the plug on 32-bit by greggyb on Thursday February 09, 2017 @03:51PM (#13610839)
Pythons 4, 5, 6..., N will feasibly exist in the future. Why should Python 2 forever retain the executable name 'python'?
Re:Arch Linux pulls the plug on 32-bit by michaelmrose on Sunday February 12, 2017 @01:32AM (#13627111)
Eventually logically python 2 will be little used enough to be dropped and there may not be a python4 any time in the foreseeable future so perhaps you are just borrowing trouble.
Re:Arch Linux pulls the plug on 32-bit by marvy on Thursday February 09, 2017 @07:48PM (#13612432)
If Py4 is compatible with Py3, why bump the version number? If not, you better be careful which one you get!
Re:Arch Linux pulls the plug on 32-bit by boznz on Thursday February 09, 2017 @02:53PM (#13610293)
Lets hope Python 4 comes out soon so you guys can unite against the real enemy :-)
Re:Arch Linux pulls the plug on 32-bit by djsumdog on Thursday February 09, 2017 @01:53PM (#13609681)
Ubuntu does this now with Xenial.
Re:Arch Linux pulls the plug on 32-bit by pseudalopex on Thursday February 09, 2017 @02:21PM (#13609964)
Ubuntu uses Python 3 where possible, but /usr/bin/python is 2.7.
Re:Arch Linux pulls the plug on 32-bit by Asooka on Thursday February 09, 2017 @09:41AM (#13607194)
In fairness, this was back when we all thought Py3 would take over Py2 "any day now"™
Re:Arch Linux pulls the plug on 32-bit by avian on Thursday February 09, 2017 @09:46AM (#13607236)
"back then" the decision made even less sense, since a lot of efforts to make Python 3.x code backwards compatible did not happen yet. Like for instance the u'' literals introduced in 3.3.
Re:Arch Linux pulls the plug on 32-bit by seiferteric on Thursday February 09, 2017 @01:10PM (#13609260)
Why didn't python just opt to have the python 2 and 3 interpreter in the same binary? It could have detected if a file was python3 with a magic comment or something (overridable with a -2/-3 option), or otherwise assume python2. Or even simpler, have a wrapper script just called 'python' that checks if the file you are calling is python2 or 3 and calls the appropriate binary.
Re:Arch Linux pulls the plug on 32-bit by Zancarius on Thursday February 09, 2017 @01:32PM (#13609484)
"Explicit is better than implicit." is the Python mantra. Although it's not often taken as literally as some would like, I think it's appropriate in this case. After all, magic detection sounds good on the surface, but it's liable to be problematic and bug ridden.

You don't need magical file-specific juju or a wrapper script on most systems these days to select the Python version. Just call python2 or python3 as appropriate; the symlinks usually exist depending on packager, distro, etc. The problem up-thread appears to be with calling the default `python` on $PATH. Sometimes it points to Python 2.x, sometimes it points to Python 3.x. Sometimes it might even change (Gentoo's eselect, IIRC). I don't see the problem being explicit with which version is needed in the script's hashbang. Python 3 isn't exactly new, and I remember when Gentoo and Arch both migrated to versioned symlinks. It wasn't that painful.