News for hackers. Stuff that matters.
Re:Escaping from Anaconda's Stranglehold on macOS by apetresc on Sunday September 01, 2024 @05:15PM (#41420455)
> 10. What Changes When You Are Free

> You will not be forced to work inside something known as a “virtual environment.”

Oof, this terrible advice cancels out an otherwise reasonable post. Beginners who don't know what they're doing are the last people who should be `pip install -r requirements.txt`-ing into the system Python the way this article is recommending. That's not only going to make working on multiple projects nearly impossible (especially for the kind of beginning students who get recommended Anaconda, which are almost always the Data Science-y crowd using NumPy, Pandas, Scikit, etc, which are notoriously finicky with version conflicts), but it stands a good chance of breaking other Python-based system utilities in completely opaque ways. This sort of advice can fubar a naive user's entire workflow.

I know virtualenvs suck to explain to people, but in my opinion it needs to be done before you ever tell them about `pip install`.

Re:Escaping from Anaconda's Stranglehold on macOS by dumbo-octopus on Sunday September 01, 2024 @05:47PM (#41420671)
Yes. I recently had to debug my gf’s entire work computer setup getting fubar’d from installing a pip package at a user level that conflicted with the name of some package used internally by the 10k+/seat/year software the company devices have installed. Proximal cause? The installer “helpfully” failed over to installing at the user level when she didn’t have permissions to write to the place she told it to. Root cause of course is that million dollar fancy fancy enterprise software somehow being unable to give itself an isolated python package namespace, or provide any relevant errors when conflicts occurred.

The python packaging system has to be used in case studies of worst design decisions of all time.

Re:Escaping from Anaconda's Stranglehold on macOS by adolph on Sunday September 01, 2024 @06:15PM (#41420866)
> package used internally by the 10k+/seat/year software the company devices have installed

Sounds like a skill issue that shouldn’t be seen in “10k+/seat/year software.” I wonder what other fails are in it.

Re:Escaping from Anaconda's Stranglehold on macOS by dumbo-octopus on Sunday September 01, 2024 @06:23PM (#41420922)
Well it constantly crashes and requires every shop to develop their own half-broken way of interfacing between it and other industry-standard software, but at least ~half the pixels you’ve seen in any modern movie or tv show have been touched by it at one point or another. So that’s interesting.
Re:Escaping from Anaconda's Stranglehold on macOS by geysersam on Sunday September 01, 2024 @06:40PM (#41420999)
Thats extremely fascinating. What's it used for?
Re:Escaping from Anaconda's Stranglehold on macOS by dumbo-octopus on Sunday September 01, 2024 @09:00PM (#41421636)
By the books Digital Compositing, but Editors also gravitate towards it for generalist work.
Re:Escaping from Anaconda's Stranglehold on macOS by AtlasBarfed on Sunday September 01, 2024 @07:40PM (#41421270)
Python has dogshit package management that makes java look well thought out and sane.
Re:Escaping from Anaconda's Stranglehold on macOS by echelon on Sunday September 01, 2024 @11:12PM (#41422309)
PYTHON @&$!!

The Python community needs to solve this ASAP. This almost weekly pain point has turned a language I used to love in college into one of my most despised languages. The fact that the ML community uses this broken platform is infuriating.

Make a Python version 4 that focuses only on fixing the packaging. 100% per-project hermeticy. No global packages whatsoever. Solve just this issue and bring the entire ecosystem on board. Kill all the various virtualenvs, the anacondas, global packages. All of it.

Learn from Rust/Cargo. That project does it mostly right (sans lack of namespaces and reproducible builds).

Re:Escaping from Anaconda's Stranglehold on macOS by mr_toad on Sunday September 01, 2024 @11:42PM (#41422402)
Wouldn’t this end up like the XKCD standards cartoon. You’d have yet another Python package system and even more fragmentation.

https://xkcd.com/927/

Re:Escaping from Anaconda's Stranglehold on macOS by ethbr1 on Monday September 02, 2024 @12:14AM (#41422525)
If Python users weren't comfortable living in that reality already, they wouldn't still be Python users.
Re:Escaping from Anaconda's Stranglehold on macOS by Eisenstein on Sunday September 01, 2024 @11:28PM (#41422361)
It took 10 years to force a breaking migration from Python 2 to Python 3 in order to handle Unicode...

I have resigned myself to repeatedly smashing my keyboard on the desk until it rains keys in blind rage induced frustration to let off steam and then calmly creating an issue in the appropriate repo instead of allowing even a modicum of hope that this will ever be fixed systemically.

Re:Escaping from Anaconda's Stranglehold on macOS by superlopuh on Monday September 02, 2024 @03:14AM (#41423278)
As far as I'm concerned, this is the fix: https://github.com/astral-sh/uv
Re:Escaping from Anaconda's Stranglehold on macOS by jampekka on Monday September 02, 2024 @06:32AM (#41424223)
It's never going to be solved. The mess is deeply ingrained in the Python development culture.

https://peps.python.org/pep-0582/

Re:Escaping from Anaconda's Stranglehold on macOS by 1propionyl on Monday September 02, 2024 @12:15AM (#41422527)
> that makes java look well thought out and sane

That's not hard. Java's package management is well thought out and sane.

Re:Escaping from Anaconda's Stranglehold on macOS by lupire on Monday September 02, 2024 @09:19AM (#41425171)
Well, it's still PATH based, which is much worse than virtual environments (directory scoped) or Nix (well-versioned sharing).
Re:Escaping from Anaconda's Stranglehold on macOS by pavlov on Monday September 02, 2024 @03:26AM (#41423320)
Python's package management makes JavaScript look well thought out and sane.

The bar is so low, you have to excavate it from a landfill of rotten node_modules...

Re:Escaping from Anaconda's Stranglehold on macOS by jampekka on Monday September 02, 2024 @06:31AM (#41424214)
Node.js has (or at least had before the ES6 mess) the sanest and best thought out package manager out there.
Re:Escaping from Anaconda's Stranglehold on macOS by dumbo-octopus on Monday September 02, 2024 @12:10PM (#41426615)
NPM’s original sin was making package installation and general management so painless that folks installed micro packages for everything. Can’t really fault the software for that.

The issue was, as you say, the introduction of ESM. It used to be that you required modules one way and one way only (yes there was AMD for advanced use cases, but it was an add-on), then people felt the need to “standardize” that, no we have this mess of ESM and CJS.

Re:Escaping from Anaconda's Stranglehold on macOS by jampekka on Monday September 02, 2024 @05:20PM (#41428910)
I don't see what's wrong with micropackages.
Re:Escaping from Anaconda's Stranglehold on macOS by VHRanger on Monday September 02, 2024 @09:13PM (#41430236)
Massive attack surface. That's true as a security risk, but also as a dependency risk (eg. left-pad).
Re:Escaping from Anaconda's Stranglehold on macOS by dumbo-octopus on Tuesday September 03, 2024 @05:34AM (#41433047)
If you don’t vendor your dependencies. Which is a poor practice that is commonly associated with NPM, but is by no means a requirement of the technology.
Re:Escaping from Anaconda's Stranglehold on macOS by dumbo-octopus on Tuesday September 03, 2024 @05:46AM (#41433114)
Especially if…, I should say. Even vendored dependencies are a risk as NPM commits the additional sin of allowing the act of pulling a package onto the local machine for inspection to execute arbitrary code in the form of “postinstall” hooks.
Re:Escaping from Anaconda's Stranglehold on macOS by jampekka on Tuesday September 03, 2024 @06:35AM (#41433401)
Review and pin your direct dependencies. With transitive dependencies it doesn't differ from trusting large dependencies in general.

The alternative to micropackages has significant downsides. Pulling in extra surface and rolling your own buggy implementations while waiting for some commitee to bikeshed years on the implementation.

Making the right thing easy rather than the wrong thing hard is a lot better approach.

Re:Escaping from Anaconda's Stranglehold on macOS by dvdkon on Monday September 02, 2024 @04:42AM (#41423682)
I actually like Python's package management (with just pip and venv). It's different to all other modern solutions, taking a per-shell rather than per-project approach, but that doesn't mean worse.

The advantage is that it's less "magic" than, say, npm. No need for rules saying how to find node_modules, you just have a PYTHONPATH env variable.

The Rust and Java approach is to do everything through a build tool's CLI, and I can't complain about that. It's probably the best compromise: Less magic than npm, and more user-friendly than Python.

Re:Escaping from Anaconda's Stranglehold on macOS by jampekka on Monday September 02, 2024 @06:29AM (#41424206)
What magic is there for finding node_modules? It's in the same directory as package.json. Very similar (though simpler) to how Python modules work.

There's also usually one global package dir, but that's almost exclusively used for runnable binaries, and not even really needed when there's npx.

If you think Python packaging has less magic, you probably don't know it very deeply.

I lost all hope for Python package management to ever get better when PEP 582 got rejected for some stupid tribal reasons.

https://peps.python.org/pep-0582/

Re:Escaping from Anaconda's Stranglehold on macOS by dvdkon on Monday September 02, 2024 @07:08AM (#41424372)
I was talking more about the package lookup when running code, via e.g. `node script.js`. I think it looks in all parent directories of the CWD, or maybe of the script? It's not too complicated, but it is "more magic" IMO.

Actually building Python packages is pretty complex, but that's the case for JS too. Java avoids this by distributing compiled libraries.

Re:Escaping from Anaconda's Stranglehold on macOS by jampekka on Monday September 02, 2024 @07:24AM (#41424447)
It looks up from script.js'd directory or the earliest parent with node_modules. Not CWD. A lot like "import somemodule" in script.py tries to import somemodule.py file from the directory script.py is in.

Traversing to the parent is especially nice for scripts. In python having scripts outside the module directory is quite painful.

Gyp is a lot easier than Python setup.py. The "easy" Java packages are comparable to pure Python/pure JS packages. With e.g. C bindings JNI/Java packaging is a horrid pain.

Re:Escaping from Anaconda's Stranglehold on macOS by LarsDu88 on Monday September 02, 2024 @12:03AM (#41422474)
Re:Escaping from Anaconda's Stranglehold on macOS by BugsJustFindMe on Sunday September 01, 2024 @08:07PM (#41421383)
> I know virtualenvs suck to explain to people, but in my opinion it needs to be done

This is because Python doesn't have versioned imports, which means you can't have multiple versions of the same package in the same environment, but I like to dream about a world where this isn't the case. If instead of import foo we had import [email protected]#optional-checksum, the Python world would be massively improved. It seems like it would be such a simple change too.

Re:Escaping from Anaconda's Stranglehold on macOS by zbentley on Sunday September 01, 2024 @09:08PM (#41421681)
Virtualenvs solve not having separate workspace-local package installations, not a lack of versioned imports. Versioned imports are not a good solution to separate installations of packages: code is harder to upgrade, cluttered, and encouraged to depend on specifics rather than contracts. There’s a reason every major language localizes their version pinning into a per-project dependencies file (which can be anywhere on the spectrum between “contracts only semver ranges” and “checksummed/vendored lockfile”). And that’s before considering the troublesome behaviors that emerge when you permit importing multiple versions of the same library in the same program, if you want that too.
Re:Escaping from Anaconda's Stranglehold on macOS by eru on Sunday September 01, 2024 @09:12PM (#41421710)
> And that’s before considering the troublesome behaviors that emerge when you permit importing multiple versions of the same library in the same program, if you want that too.

Rust has versioned imports. So you can import different versions of the same module (at least in your transitive dependencies).

Re:Escaping from Anaconda's Stranglehold on macOS by zbentley on Monday September 02, 2024 @07:32AM (#41424498)
Yep. And while that’s not something I’d call a mistake per se, it can be troublesome. What if a crate you depend on at different versions provides access to global state? What if a transitive embeds definitions from that crate into public contracts consumed by code using a different version?

Rust offers mitigations or means of detection for those cases, but they still require thought and troubleshooting when they occur. Given Python’s lack of static typing and more-likely-to-be-nonexpert user base and usage patterns, I suspect that troublesomeness would not be a value-add for the Python platform.

Re:Escaping from Anaconda's Stranglehold on macOS by eru on Tuesday September 03, 2024 @05:37AM (#41433063)
Well, for what it's worth, Rust's linter clippy likes to warn you about using different versions of the same crate. I think the warning is mostly formulated in terms of helping you reduce compile times, but your concerns about mutability might also be justified.
Re:Escaping from Anaconda's Stranglehold on macOS by abdullahkhalids on Sunday September 01, 2024 @06:31PM (#41420952)
I teach using python. You should try explaining virtual environments to students, most of them with no programming experience, or any real notion of the state of a computer system. I do, because we recommend students use them. Every class consists of endless debugging of student systems.

After that you might understand the OP's viewpoint.

Re:Escaping from Anaconda's Stranglehold on macOS by freeopinion on Sunday September 01, 2024 @07:02PM (#41421109)
After that I wonder why people think Python is a good choice to teach as a first experience in programming. Virtual environments is actually my biggest gripe with Python.

I think it is interesting to consider why Python is so popular despite such fiascos. The answers can be very informative but can also be giant red flags.

Re:Escaping from Anaconda's Stranglehold on macOS by Al-Khwarizmi on Sunday September 01, 2024 @07:22PM (#41421194)
Because every option sucks for teaching.

C/C++? Full of footguns. Java/C#? Too complicated for beginners. Pascal? Outdated. Etc.

I personally prefer C to teach first-year CS students but just as the lesser evil. A good first programming language is sorely lacking.

(Note that I'm talking about the imperative programming paradigm. The debate on whether one should start with functional programming is outside the scope of this comment).

Re:Escaping from Anaconda's Stranglehold on macOS by Spivak on Sunday September 01, 2024 @08:24PM (#41421462)
It's weird you throw out Java because they are the teaching language. They're the stick everything else is measured against. CS101 is a sea of Eclipse. Having a programming environment where all the major structures are discoverable through menus is actually pretty useful.
Re:Escaping from Anaconda's Stranglehold on macOS by dvdkon on Monday September 02, 2024 @04:20AM (#41423576)
Java is popular as a first language, but I think it's just due to its (bygone?) general popularity. I don't think a language that forces you into a one-class-per-file mindset from day one has any business being used for teaching; students should start from the basics ("commands"/function calls, if, while, for, ...) and then discover what more complex concepts do within that framework.

Java's tooling is nice, but frankly Turbo Pascal (or something at its level) is enough for beginners.

Re:Escaping from Anaconda's Stranglehold on macOS by eru on Sunday September 01, 2024 @09:09PM (#41421691)
You can teach imperative programming with OCaml or Racket, too. :)

C is indeed pretty evil, though slightly less though with modern sanitisers, so you can get a better error message than just a segfault (or silently doing the wrong thing).

Pascal isn't really more outdated than C. Especially if you use Delphi?

Python is actually fine, you can get pretty far with just the standard library, and the libraries that you can install with your Linux distribution's package manager (eg via Pacman). I do agree that package management with Python is pretty bad out-of-the-box.

Re:Escaping from Anaconda's Stranglehold on macOS by bmitc on Sunday September 01, 2024 @10:25PM (#41422127)
> Because every option sucks for teaching.

The answer is really quite simple: F#

Re:Escaping from Anaconda's Stranglehold on macOS by moi2388 on Monday September 02, 2024 @01:05AM (#41422740)
How in the world is c# too complicated? Console app, top level statements, vs code. As easy as Python. Easier because your env isn’t messed up.

If they can’t manage that they have no business doing data science or something similar to begin with

Re:Escaping from Anaconda's Stranglehold on macOS by cuddlyogre on Monday September 02, 2024 @09:56AM (#41425416)
You sound like you have been doing this for way too long to know what it takes to teach beginners.
Re:Escaping from Anaconda's Stranglehold on macOS by neonsunset on Monday September 02, 2024 @10:55AM (#41425843)
Not for complete but for beginners lightly familiar with programming it is overall not difficult to teach the basics, often easier than to stubborn seniors who want to do it their way only :)

    var start = "commtext c00\">";
    var end = "</div>";

    using var http = new HttpClient();
    var page = await http.GetStringAsync("https://news.ycombinator.com/item?id=41425416");

    var commStart = page.IndexOf(start) + start.Length;
    var commEnd = page.IndexOf(end, commStart);

    Console.WriteLine(page[commStart..commEnd]);
or

    var host = WebApplication
        .CreateBuilder()
        .Build();

    host.MapGet("/", () => "Hello World");
    host.Run("http://+:8080");
Re:Escaping from Anaconda's Stranglehold on macOS by moi2388 on Tuesday September 03, 2024 @11:55AM (#41436171)
Not at all. I still teach interns and until 2 years ago complete beginners.

I can teach them a console app with top level statements and have it running before they have installed a Python virtual environment.

Re:Escaping from Anaconda's Stranglehold on macOS by bspammer on Sunday September 01, 2024 @07:27PM (#41421210)
It’s not just the packaging situation that makes Python a bad first language. Whenever I write in Python, I find it completely impossible to stop myself from constantly making basic, beginner-level programming mistakes. Every time I miss having a compiler and strict typing that will yell at me when I’ve done something stupid.
Re:Escaping from Anaconda's Stranglehold on macOS by eru on Sunday September 01, 2024 @09:11PM (#41421704)
Modern linters and type-checkers for Python come pretty close to something usable for these situations. But it's certainly something they tacked on to the language afterwards.

I like the recent addition of (proper) pattern matching to Python.

Re:Escaping from Anaconda's Stranglehold on macOS by bmitc on Sunday September 01, 2024 @10:27PM (#41422136)
Yea, the only way to get close to programming efficiently in Python is to turn MyPy and Pylint up to 11.
Re:Escaping from Anaconda's Stranglehold on macOS by airstrike on Sunday September 01, 2024 @09:44PM (#41421884)
I switched to Rust full time about 6 months ago and recently had to write a somewhat long Python script for something else... it felt awfully quiet without the compiler yelling at me, but that also meant I was making the same silly mistakes over and over and over and over
Re:Escaping from Anaconda's Stranglehold on macOS by abdullahkhalids on Sunday September 01, 2024 @08:18PM (#41421427)
We use it because of other people use it. I would gladly switch to Julia for scientific commuting if I could.
Re:Escaping from Anaconda's Stranglehold on macOS by zaptheimpaler on Sunday September 01, 2024 @07:08PM (#41421147)
I think docker would be a much better alternative. You can just hand students a pre-built docker with everything installed and they are free to mess up the system without any consequences.
Re:Escaping from Anaconda's Stranglehold on macOS by makeitdouble on Sunday September 01, 2024 @07:57PM (#41421339)
But then you're explaining virtualization and networking to your students, how to manage interactions in and out ("I wrote a program in Txtedit, how can I run it?"), when it's started and when it's not etc.

I could see giving them a remote access to a prebuilt env (which can be containerized) to be simpler to explain.

Re:Escaping from Anaconda's Stranglehold on macOS by Daishiman on Sunday September 01, 2024 @08:52PM (#41421602)
In finding it very difficult to see what exactly is the problem with teaching a minimum of computer and operating system fundamentals to people that have to learn programming.
Re:Escaping from Anaconda's Stranglehold on macOS by abdullahkhalids on Sunday September 01, 2024 @09:20PM (#41421762)
Not every person wants to become a software engineer, anymore than every person wants to learn to become a plumber or car mechanic. The vast majority, and by vast majority I mean more than 95% of students who are taught to code are training to be scientists, mathematicians, engineers, economists, accountants, or some such. They need to run simulations to understand their field. Any pain in the way of that is a failure of those who create software for a living.
Re:Escaping from Anaconda's Stranglehold on macOS by Daishiman on Sunday September 01, 2024 @10:32PM (#41422161)
If you're going to be spending a substantial part of your waking life working on a tool, spending a few hours to get the basics right is not an unreasonable ask. And as it turns out, all non-trivial programming languages that have some sort of packaging system or module system have some version of the problems involved.

You don't see people complaining that musical instruments are unreasonably complex when that complexity usually gets solved after a couple months of training, or when someone who wants to basic woodworking has to have at least a passing knowledge of the different types of hardwoods, MDFs, and essential joinery techniques.

Hate to say, but it definitely sounds like skills issues.

Re:Escaping from Anaconda's Stranglehold on macOS by makeitdouble on Sunday September 01, 2024 @10:28PM (#41422138)
I'd argue it's a lot more than "minimum". You need them to be comfortable with the host system, and also manage the running container, which will have a different OS most of the time, and the virtualization mechanism (docker here).

Simple things like having a student access another student's web server becomes overly complicated, as you're dealing with 4 systems talking to/through each other.

Re:Escaping from Anaconda's Stranglehold on macOS by Daishiman on Sunday September 01, 2024 @10:36PM (#41422178)
But the fundamental fact is that "accessing someone else's web server" is not that trivial of an affair and has never been when you get down and dirty into the details of networking.
Re:Escaping from Anaconda's Stranglehold on macOS by makeitdouble on Sunday September 01, 2024 @10:50PM (#41422237)
Giving access to a debug server on a local network is a single command line when running natively to the host.

Sure making them understand everything is a journey, but just spawning the server is fairly quick and painless.

Re:Escaping from Anaconda's Stranglehold on macOS by BlueTemplar on Thursday September 05, 2024 @07:10PM (#41461194)
Might as well use Guix at that point, since that kind of OS-based dependency management seems to be the future.
Re:Escaping from Anaconda's Stranglehold on macOS by bigger_cheese on Sunday September 01, 2024 @09:19PM (#41421750)
Not sure how helpful this advice is for students I'm in my late 30's have programming experience but had never touched Python until about 6 months ago.

What made Virtual environments work for me was switching to a different Editor. I use Visual Studio Code currently it works well with Virtual Environments.

When you first create a new python file in a given "project folder" it prompts you to create a new Venv and when you switch project folders it remembers and restores the Venv for each project.

One of my work colleagues pointed me to VSCode - it streamlined a lot of python things for me.

If your students are disciplined about creating a new folder for each project managing virtual environments vscode could help them.

Only issue I have is my work office has a corporate proxy setup and pip needs certificate to connect (and if I work remotely I have to turn this setting off) I wrote a shell script to toggle between the two proxy settings. Not sure if university will have the same issue proxy issues but if so this would certainly be a pain point for many students.

Re:Escaping from Anaconda's Stranglehold on macOS by promer on Sunday September 01, 2024 @11:34PM (#41422379)
This is a suggestion that deserves serious consideration. I started using VS Code after I had learned all the basics so I don't know what it feels like to be a new learner who is getting started by using VS Code. But it sounds like it worked for you. (And on campus, the proxy should not be an issue.)

It does answer the question of which text editor to use when the time comes to teach them about editing text files. For a course, it helps that it is free and available on both macOS and Windows.

I have a vague recollection that at some point I was testing without an install of Python and after I selected the Python extension for VS Code, it offered to install one for me. (I don't recall if this was on macOS or Windows and memory could be playing a trick on me.) But in any case, the people behind VS Code do seem to be trying very hard to make it easy for someone who is getting started.

I am a little uneasy about the nudges to use Copilot, but on balance, it might offer a better path for students who are getting started.

Re:Escaping from Anaconda's Stranglehold on macOS by bmitc on Sunday September 01, 2024 @10:24PM (#41422121)
Then why teach Python? All of these issues are because Python is a ridiculously poorly designed language which is still having new features added to it. It's generally a mess.
Re:Escaping from Anaconda's Stranglehold on macOS by regularfry on Monday September 02, 2024 @09:21AM (#41425183)
The issues are with one particular facet of poor design and management. The rest of it is manageable, if distasteful, and worth the effort for the ecosystem (in general).
Re:Escaping from Anaconda's Stranglehold on macOS by bmitc on Monday September 02, 2024 @01:57PM (#41427465)
There really is more than one particular face of poor design in Python. I honestly remain confused why people use and love Python so much. It very much says something.
Re:Escaping from Anaconda's Stranglehold on macOS by regularfry on Monday September 02, 2024 @07:36PM (#41429692)
Oh, there are many. There are just comparatively few that you can point at as being quite as directly responsible for dev misery as thé packaging system.
Re:Escaping from Anaconda's Stranglehold on macOS by bmitc on Tuesday September 03, 2024 @05:52PM (#41439663)
Lol. I see where you're coming from.
Re:Escaping from Anaconda's Stranglehold on macOS by Eduard on Sunday September 01, 2024 @10:36PM (#41422180)
> You should try explaining virtual environments to students

can you? Because I can't.

e.g. "What is the difference between venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, pipenv, etc?" ( https://stackoverflow.com/questions/41573587/what-is-the-dif... )

what is the difference between conda and anaconda?

etc etc

Re:Escaping from Anaconda's Stranglehold on macOS by rahimnathwani on Sunday September 01, 2024 @10:43PM (#41422213)
That's exactly GP's point.
Re:Escaping from Anaconda's Stranglehold on macOS by justinclift on Monday September 02, 2024 @03:25AM (#41423318)
Reckon they'd understand using something like asdf instead?

https://asdf-vm.com

The projects I work with have moved to it because it handles multiple languages (not just Python). It's made things easy to keep "the correct version of a language" that each project needs.

So far so good, anyway. :)

Re:Escaping from Anaconda's Stranglehold on macOS by whywhywhywhy on Monday September 02, 2024 @05:43AM (#41424000)
You’re swapping something that can be feasibly debugged or at least easily burnt and reformed for something that’s impossible to untangle once they’ve done two projects with different or conflicting deps into the base environment.
Re:Escaping from Anaconda's Stranglehold on macOS by orlp on Sunday September 01, 2024 @07:07PM (#41421137)
What I do on every dev machine is build Python from source and install it to ~/.localpython and put it in my path.

Then I can pip install away care-free, and quickly run scripts / repls without having to activate an environment.

Re:Escaping from Anaconda's Stranglehold on macOS by eru on Sunday September 01, 2024 @09:12PM (#41421715)
Why does it need to be built from source for that?
Re:Escaping from Anaconda's Stranglehold on macOS by orlp on Monday September 02, 2024 @02:20AM (#41423054)
It doesn't, but it's typically the simplest (for me). It also means I get exactly the version I want compiled with the best optimizations for that machine.
Re:Escaping from Anaconda's Stranglehold on macOS by eru on Monday September 02, 2024 @03:52AM (#41423462)
Thanks, that makes sense.
Re:Escaping from Anaconda's Stranglehold on macOS by m0llusk on Tuesday September 03, 2024 @03:21PM (#41438216)
Typically building from source is done to include optional components such as with the ./configure --with-pydebug option. With many projects make doc is an option because so many prefer to skip that part of the build in favor of online documentation.
Re:Escaping from Anaconda's Stranglehold on macOS by sevensor on Sunday September 01, 2024 @11:12PM (#41422310)
Building Python is so quick and easy I’ve built an interpreter per project in some circumstances. Sometimes more if I want to make sure it’s compatible with multiple Python versions.
Re:Escaping from Anaconda's Stranglehold on macOS by hkdfgdfg on Sunday September 01, 2024 @05:50PM (#41420687)
I have never broken my Python installation on Windows (official installer) despite recklessly installing anything. If I had broken it, I would have just uninstalled and reinstalled.

While not familiar with macOS (topic of this article), I think the Mac installer works the same.

On Debian of course you can cause great damage because the system Python is used for critical system functions. This is silly, I think the system python should be an isolated install in /usr/sbin. Better yet, move back to Perl for the system.

Re:Escaping from Anaconda's Stranglehold on macOS by rbanffy on Sunday September 01, 2024 @06:43PM (#41421020)
You won’t break a machine by installing Python packages on an OS that never uses Python for anything.
Re:Escaping from Anaconda's Stranglehold on macOS by analog31 on Monday September 02, 2024 @04:04PM (#41428368)
This is a good point and something I hadn't thought about. I'm also a happy Windows user, and have never had a problem with Python installations. But like you say, Python isn't baked into the system.

I do use one piece of commercial software that includes Python. You can see the installer saying "installing Python." I suppose it should fall on the OS and software vendors to not abuse the infrastructure... "to whom much is given, much is expected," but maybe too much to expect.

Re:Escaping from Anaconda's Stranglehold on macOS by rbanffy on Monday September 02, 2024 @04:28PM (#41428536)
I remember that, in the past at least, one could completely destroy a Red Hat install by messing too much with the system Python.
Re:Escaping from Anaconda's Stranglehold on macOS by analog31 on Monday September 02, 2024 @04:36PM (#41428600)
I guess "Python is my operating system" has its unintended consequences. ;-)
Re:Escaping from Anaconda's Stranglehold on macOS by swiftcoder on Monday September 02, 2024 @04:14AM (#41423549)
To be clear, there's absolutely nothing broken about the system Python in the article. There is just a shell alias causing the anaconda version of Python to be launched instead when you type `python3` at the command prompt...
Re:Escaping from Anaconda's Stranglehold on macOS by nox101 on Sunday September 01, 2024 @11:38PM (#41422389)
IMO what needs to happen is python needs to get rid of system installs and only work via venv. It should check the current folder for some .venv or something and auto-config itself to the current directory (or complain that you need to config with a messages like "python not configured for current folder. Run 'source bin/activate' to configure. Honestly thought that could be so much better.

The entire problem of python an venv IMO comes from it not being the default.

Re:Escaping from Anaconda's Stranglehold on macOS by polemic on Monday September 02, 2024 @12:24AM (#41422561)
Python developers agree. See PEP 668 – Marking Python base environments as “externally managed” – https://peps.python.org/pep-0668/
Re:Escaping from Anaconda's Stranglehold on macOS by brylie on Monday September 02, 2024 @01:30AM (#41422853)
Re:Escaping from Anaconda's Stranglehold on macOS by ajb on Monday September 02, 2024 @01:57AM (#41422957)
Hmm that still recommends that distros allow admins to install to /usr/local, albeit in such a way that it at least can't break the OS.

IMO the idea that a 'Linux admin' is better informed than a 'Linux user' is increasingly anachronistic. In most cases the admin is just the user running sudo. I'd suggest that such functionality should be enabled by installing some kind of OS package rather than being the default

Re:Escaping from Anaconda's Stranglehold on macOS by jampekka on Monday September 02, 2024 @06:51AM (#41424308)
Yet another hack on hack. Venv was the wrong fix/hack and now the package management is cascading into even more madness.
Re:Escaping from Anaconda's Stranglehold on macOS by lhfacx on Monday September 02, 2024 @08:34AM (#41424899)
This is insane. Apparently (at least on Debian) this can be circumvented by putting this in ~/.config/pip/pip.conf:

  [global]
  break-system-packages = true
I would have preferred single-version-externally-managed to keep fond memories of setuptools alive.

It becomes increasingly impossible to track down home directory pollution and config files in Python. Next step will be a Python registry on Linux. How about:

  regedit VENV=/home/sjw/venv42 KEYWORD=single-version-externally-managed DWORD=0xbadbee
Re:Escaping from Anaconda's Stranglehold on macOS by delusional on Monday September 02, 2024 @12:36AM (#41422611)
This is already the case on Arch, where pip will yell at you if you try to install anything to either the user or system score.
Re:Escaping from Anaconda's Stranglehold on macOS by mrweasel on Monday September 02, 2024 @02:49AM (#41423178)
That depends very much on what you're doing. For production I'd like to use the OS package manager to install my Python dependencies, and move the responsibility of patching to the OS.

For workstations, absolutely, go with virtual environments, it's the only way to go. One concern I've seen from some in the machine learning space is that rebuilding a virtual environment, for example if macOS upgrades the Python version, takes hours. That can be solved by using pyenv, then you can have multiple versions of Python and be free of Anaconda. I primarily use pyenv to be sure that I have the same Python version that ships with the server OS on my laptop.

Re:Escaping from Anaconda's Stranglehold on macOS by flessner on Monday September 02, 2024 @03:17AM (#41423288)
I have been out of the loop with Python for a couple years at this point, but how could this get so bad?

If you are looking into a system you are unfamiliar with, where do you look first?... pip? pipx? homebrew?... or is it in anaconda? pyenv?... must be in the os package manager... apt? pacman?

Honestly, Maven and NPM look great compared to this mess.

Re:Escaping from Anaconda's Stranglehold on macOS by mrweasel on Monday September 02, 2024 @03:39AM (#41423397)
NPM to me is the great example of a package manager that worse than anything the Python community has come up with, but that's subjective I think.

Python isn't as bad as people make it out to be. There are some issue that you will run into if your project/code base becomes really large, but that's not an issue for most people. The vast majority can get just use python -mvenv .venv to set up a virtual environment and install in that using pip.

Next step up is you need specific versions of Python, so you switch to pyenv, which functions mostly like the built in virtualenv.

Then you have the special cases where you need to lock dependencies much hard than pip can do and you use poetry, pip is to slow so you use pipx. Those are edge cases to be honest. That's not to say that they aren't solving very real problem, but mostly you don't need it.

It would be great if there was one tools that could do it all, lock the Python version, do quick dependency resolution and lock them down.

So far Python has opted to split the problem: Tools for locking the Python version, tools for creating separate environments and tools for managing packages. There's a lot of crossover in the first two, but you can pretty much mix and match virtual environment tools and package managers anyway you like. I think that's pretty unique.

Re:Escaping from Anaconda's Stranglehold on macOS by flessner on Monday September 02, 2024 @04:29AM (#41423627)
It's definitely unique, but the "out of the box" experience is suffering for it in my opinion.

I actually recently had to fix something small in a Python project, pip refused to work because of homebrew, homebrew didn't have the dependencies and directed me to pipx, pipx finally worked - It was a strange experience.

And for the record NPM mostly has a bad reputation because of its past... nowadays it's perfectly usable and can lock dependencies and node version "out of the box".

Re:Escaping from Anaconda's Stranglehold on macOS by mrweasel on Monday September 02, 2024 @04:35AM (#41423649)
> It's definitely unique, but the "out of the box" experience is suffering for it

The tooling that ships with Python could be much better, I'd agree with that. You can go pretty far with venv and pip, but just the python3 -mvenv .venv isn't exactly a great example of user friendliness at work.

Re:Escaping from Anaconda's Stranglehold on macOS by jampekka on Monday September 02, 2024 @06:55AM (#41424324)
What's your problem with NPM? It just works, it's easy to package for, it handles multiple version dependencies elegantly, with pnpm it doesn't trash your disk and is really fast.

(P)NPM and CJS is the best package/dependency management there currently is.

Re:Escaping from Anaconda's Stranglehold on macOS by eddyg on Monday September 02, 2024 @12:59PM (#41427011)
I always set:

    export PIP_REQUIRE_VIRTUALENV=true
Re:Escaping from Anaconda's Stranglehold on macOS by promer on Monday September 02, 2024 @12:22AM (#41422551)
I don't see how it could be correct to say that I'm recommending that students pip install into the system Python.

On macOS, there hasn't been a system Python since 12.3 came out in Jan of 2022. And as far as I know, there was never a system Python on Windows.

Re:Escaping from Anaconda's Stranglehold on macOS by swiftcoder on Monday September 02, 2024 @04:21AM (#41423579)
> On macOS, there hasn't been a system Python since 12.3 came out in Jan of 2022

It may not be pre-installed, but if you install Xcode it will also install python3 in /usr/bin/ for you...

Re:Escaping from Anaconda's Stranglehold on macOS by hwgern on Monday September 02, 2024 @06:23AM (#41424180)
Xcode users should probably be able to fix breakage and that /usr/bin/python is not used by MacOS for critical tasks.
Re:Escaping from Anaconda's Stranglehold on macOS by promer on Tuesday September 03, 2024 @12:22AM (#41431242)
The assertion that seemed incorrect was that I was recommending an option that would install libraries to the system Python.

Xcode does put a binary with the name python3 into `/usr/bin`. This is not a system Python, but set that aside.

Can you explain how someone might pip install libraries to this instance of Python?

Re:Escaping from Anaconda's Stranglehold on macOS by swiftcoder on Tuesday September 03, 2024 @08:23AM (#41434251)
Once Xcode's python is installed, `pip3 install ...` will install libraries to that python installation (assuming you haven't aliased pip3 to another installation, as it appears the anaconda installer did)
Re:Escaping from Anaconda's Stranglehold on macOS by promer on Tuesday September 03, 2024 @10:56AM (#41435577)
The install by xcode of a python3 binary in `/usr/bin` does not come with its own copy of pip3.

So either your command will fail to install or it will use a version of pip that is on PATH but was installed by some other version of python. So in either case you can't install a library into what you are mistakenly calling a system python.

So the original assertion was not true and you have not been able to make up an ex post justification for it.

Re:Escaping from Anaconda's Stranglehold on macOS by swiftcoder on Tuesday September 03, 2024 @01:45PM (#41437261)
I have a `/usr/bin/pip3`, and it appears to have been installed at the same time as the rest of XCode. Not sure what's up with your install:

  % ls -la /usr/bin/(pip3|python3|clang)
  -rwxr-xr-x  77 root  wheel  119008  4 Aug 12:31 /usr/bin/clang
  -rwxr-xr-x  77 root  wheel  119008  4 Aug 12:31 /usr/bin/pip3
  -rwxr-xr-x  77 root  wheel  119008  4 Aug 12:31 /usr/bin/python3
Re:Escaping from Anaconda's Stranglehold on macOS by promer on Tuesday September 03, 2024 @03:16PM (#41438166)
Yes, you are correct. pip3 is installed by XCode.

I was having trouble understanding the scenario you seem to have in mind because it never occurred to me that someone would try to run Python without doing an install from python.org as I explicitly recommend.

If someone does install a version of Python from python.org, it puts the bin folder for that version first on PATH via this line in .zprofile:

PATH="/Library/Frameworks/Python.framework/Versions/3.11/bin:${PATH}" export PATH

It also puts a symlink for python3 and pip3 into the `/usr/local/bin` folder that comes ahead of `/usr/bin`.

So if the user runs

`pip3 install ...`

it will not find the version in `/usr/bin` because there will be two other directories ahead of it on the path that have an instance of `pip3`.

As an aside, if they do the improbable and run something like

`/usr/bin/pip3 install ...`

or if they do not have any Python from python.org installed and run

`pip3 install`

what they will end up with is a user-install that puts libraries in

`~/Library/`

because at least on an Apple Silicon mac, pip can't write to `/usr/bin`. This fallback to a user-install is confusing to people who encounter it, but it is very different from "installing to the system python."

To summarize, it is exceedingly unlikely that a student who is not comfortable running commands from the terminal is going make the mistake you seem to be worried about and end up with libraries in the user-install location:

1. They do not install an official Python even though that is exactly what I recommend.

2. They do install XCode or XCode Command Line tools, then try to use `pip3 install ...`

Re:Escaping from Anaconda's Stranglehold on macOS by djaouen on Monday September 02, 2024 @01:29AM (#41422848)
You are wrong: this was not a reasonable post.