The python packaging system has to be used in case studies of worst design decisions of all time.
Sounds like a skill issue that shouldn’t be seen in “10k+/seat/year software.” I wonder what other fails are in it.
The Python community needs to solve this ASAP. This almost weekly pain point has turned a language I used to love in college into one of my most despised languages. The fact that the ML community uses this broken platform is infuriating.
Make a Python version 4 that focuses only on fixing the packaging. 100% per-project hermeticy. No global packages whatsoever. Solve just this issue and bring the entire ecosystem on board. Kill all the various virtualenvs, the anacondas, global packages. All of it.
Learn from Rust/Cargo. That project does it mostly right (sans lack of namespaces and reproducible builds).
I have resigned myself to repeatedly smashing my keyboard on the desk until it rains keys in blind rage induced frustration to let off steam and then calmly creating an issue in the appropriate repo instead of allowing even a modicum of hope that this will ever be fixed systemically.
That's not hard. Java's package management is well thought out and sane.
The bar is so low, you have to excavate it from a landfill of rotten node_modules...
The issue was, as you say, the introduction of ESM. It used to be that you required modules one way and one way only (yes there was AMD for advanced use cases, but it was an add-on), then people felt the need to “standardize” that, no we have this mess of ESM and CJS.
The alternative to micropackages has significant downsides. Pulling in extra surface and rolling your own buggy implementations while waiting for some commitee to bikeshed years on the implementation.
Making the right thing easy rather than the wrong thing hard is a lot better approach.
The advantage is that it's less "magic" than, say, npm. No need for rules saying how to find node_modules, you just have a PYTHONPATH env variable.
The Rust and Java approach is to do everything through a build tool's CLI, and I can't complain about that. It's probably the best compromise: Less magic than npm, and more user-friendly than Python.
There's also usually one global package dir, but that's almost exclusively used for runnable binaries, and not even really needed when there's npx.
If you think Python packaging has less magic, you probably don't know it very deeply.
I lost all hope for Python package management to ever get better when PEP 582 got rejected for some stupid tribal reasons.
Actually building Python packages is pretty complex, but that's the case for JS too. Java avoids this by distributing compiled libraries.
Traversing to the parent is especially nice for scripts. In python having scripts outside the module directory is quite painful.
Gyp is a lot easier than Python setup.py. The "easy" Java packages are comparable to pure Python/pure JS packages. With e.g. C bindings JNI/Java packaging is a horrid pain.
This is because Python doesn't have versioned imports, which means you can't have multiple versions of the same package in the same environment, but I like to dream about a world where this isn't the case. If instead of import foo we had import [email protected]#optional-checksum, the Python world would be massively improved. It seems like it would be such a simple change too.
Rust has versioned imports. So you can import different versions of the same module (at least in your transitive dependencies).
Rust offers mitigations or means of detection for those cases, but they still require thought and troubleshooting when they occur. Given Python’s lack of static typing and more-likely-to-be-nonexpert user base and usage patterns, I suspect that troublesomeness would not be a value-add for the Python platform.
After that you might understand the OP's viewpoint.
I think it is interesting to consider why Python is so popular despite such fiascos. The answers can be very informative but can also be giant red flags.
C/C++? Full of footguns. Java/C#? Too complicated for beginners. Pascal? Outdated. Etc.
I personally prefer C to teach first-year CS students but just as the lesser evil. A good first programming language is sorely lacking.
(Note that I'm talking about the imperative programming paradigm. The debate on whether one should start with functional programming is outside the scope of this comment).
Java's tooling is nice, but frankly Turbo Pascal (or something at its level) is enough for beginners.
C is indeed pretty evil, though slightly less though with modern sanitisers, so you can get a better error message than just a segfault (or silently doing the wrong thing).
Pascal isn't really more outdated than C. Especially if you use Delphi?
Python is actually fine, you can get pretty far with just the standard library, and the libraries that you can install with your Linux distribution's package manager (eg via Pacman). I do agree that package management with Python is pretty bad out-of-the-box.
The answer is really quite simple: F#
If they can’t manage that they have no business doing data science or something similar to begin with
var start = "commtext c00\">";
var end = "</div>";
using var http = new HttpClient();
var page = await http.GetStringAsync("https://news.ycombinator.com/item?id=41425416");
var commStart = page.IndexOf(start) + start.Length;
var commEnd = page.IndexOf(end, commStart);
Console.WriteLine(page[commStart..commEnd]);
or var host = WebApplication
.CreateBuilder()
.Build();
host.MapGet("/", () => "Hello World");
host.Run("http://+:8080");I can teach them a console app with top level statements and have it running before they have installed a Python virtual environment.
I like the recent addition of (proper) pattern matching to Python.
I could see giving them a remote access to a prebuilt env (which can be containerized) to be simpler to explain.
You don't see people complaining that musical instruments are unreasonably complex when that complexity usually gets solved after a couple months of training, or when someone who wants to basic woodworking has to have at least a passing knowledge of the different types of hardwoods, MDFs, and essential joinery techniques.
Hate to say, but it definitely sounds like skills issues.
Simple things like having a student access another student's web server becomes overly complicated, as you're dealing with 4 systems talking to/through each other.
Sure making them understand everything is a journey, but just spawning the server is fairly quick and painless.
What made Virtual environments work for me was switching to a different Editor. I use Visual Studio Code currently it works well with Virtual Environments.
When you first create a new python file in a given "project folder" it prompts you to create a new Venv and when you switch project folders it remembers and restores the Venv for each project.
One of my work colleagues pointed me to VSCode - it streamlined a lot of python things for me.
If your students are disciplined about creating a new folder for each project managing virtual environments vscode could help them.
Only issue I have is my work office has a corporate proxy setup and pip needs certificate to connect (and if I work remotely I have to turn this setting off) I wrote a shell script to toggle between the two proxy settings. Not sure if university will have the same issue proxy issues but if so this would certainly be a pain point for many students.
It does answer the question of which text editor to use when the time comes to teach them about editing text files. For a course, it helps that it is free and available on both macOS and Windows.
I have a vague recollection that at some point I was testing without an install of Python and after I selected the Python extension for VS Code, it offered to install one for me. (I don't recall if this was on macOS or Windows and memory could be playing a trick on me.) But in any case, the people behind VS Code do seem to be trying very hard to make it easy for someone who is getting started.
I am a little uneasy about the nudges to use Copilot, but on balance, it might offer a better path for students who are getting started.
can you? Because I can't.
e.g. "What is the difference between venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, pipenv, etc?" ( https://stackoverflow.com/questions/41573587/what-is-the-dif... )
what is the difference between conda and anaconda?
etc etc
The projects I work with have moved to it because it handles multiple languages (not just Python). It's made things easy to keep "the correct version of a language" that each project needs.
So far so good, anyway. :)
Then I can pip install away care-free, and quickly run scripts / repls without having to activate an environment.
While not familiar with macOS (topic of this article), I think the Mac installer works the same.
On Debian of course you can cause great damage because the system Python is used for critical system functions. This is silly, I think the system python should be an isolated install in /usr/sbin. Better yet, move back to Perl for the system.
I do use one piece of commercial software that includes Python. You can see the installer saying "installing Python." I suppose it should fall on the OS and software vendors to not abuse the infrastructure... "to whom much is given, much is expected," but maybe too much to expect.
The entire problem of python an venv IMO comes from it not being the default.
https://packaging.python.org/en/latest/specifications/extern...
IMO the idea that a 'Linux admin' is better informed than a 'Linux user' is increasingly anachronistic. In most cases the admin is just the user running sudo. I'd suggest that such functionality should be enabled by installing some kind of OS package rather than being the default
[global]
break-system-packages = true
I would have preferred single-version-externally-managed to keep fond memories of setuptools alive.It becomes increasingly impossible to track down home directory pollution and config files in Python. Next step will be a Python registry on Linux. How about:
regedit VENV=/home/sjw/venv42 KEYWORD=single-version-externally-managed DWORD=0xbadbeeFor workstations, absolutely, go with virtual environments, it's the only way to go. One concern I've seen from some in the machine learning space is that rebuilding a virtual environment, for example if macOS upgrades the Python version, takes hours. That can be solved by using pyenv, then you can have multiple versions of Python and be free of Anaconda. I primarily use pyenv to be sure that I have the same Python version that ships with the server OS on my laptop.
If you are looking into a system you are unfamiliar with, where do you look first?... pip? pipx? homebrew?... or is it in anaconda? pyenv?... must be in the os package manager... apt? pacman?
Honestly, Maven and NPM look great compared to this mess.
Python isn't as bad as people make it out to be. There are some issue that you will run into if your project/code base becomes really large, but that's not an issue for most people. The vast majority can get just use python -mvenv .venv to set up a virtual environment and install in that using pip.
Next step up is you need specific versions of Python, so you switch to pyenv, which functions mostly like the built in virtualenv.
Then you have the special cases where you need to lock dependencies much hard than pip can do and you use poetry, pip is to slow so you use pipx. Those are edge cases to be honest. That's not to say that they aren't solving very real problem, but mostly you don't need it.
It would be great if there was one tools that could do it all, lock the Python version, do quick dependency resolution and lock them down.
So far Python has opted to split the problem: Tools for locking the Python version, tools for creating separate environments and tools for managing packages. There's a lot of crossover in the first two, but you can pretty much mix and match virtual environment tools and package managers anyway you like. I think that's pretty unique.
I actually recently had to fix something small in a Python project, pip refused to work because of homebrew, homebrew didn't have the dependencies and directed me to pipx, pipx finally worked - It was a strange experience.
And for the record NPM mostly has a bad reputation because of its past... nowadays it's perfectly usable and can lock dependencies and node version "out of the box".
The tooling that ships with Python could be much better, I'd agree with that. You can go pretty far with venv and pip, but just the python3 -mvenv .venv isn't exactly a great example of user friendliness at work.
(P)NPM and CJS is the best package/dependency management there currently is.
export PIP_REQUIRE_VIRTUALENV=trueOn macOS, there hasn't been a system Python since 12.3 came out in Jan of 2022. And as far as I know, there was never a system Python on Windows.
It may not be pre-installed, but if you install Xcode it will also install python3 in /usr/bin/ for you...
Xcode does put a binary with the name python3 into `/usr/bin`. This is not a system Python, but set that aside.
Can you explain how someone might pip install libraries to this instance of Python?
So either your command will fail to install or it will use a version of pip that is on PATH but was installed by some other version of python. So in either case you can't install a library into what you are mistakenly calling a system python.
So the original assertion was not true and you have not been able to make up an ex post justification for it.
% ls -la /usr/bin/(pip3|python3|clang)
-rwxr-xr-x 77 root wheel 119008 4 Aug 12:31 /usr/bin/clang
-rwxr-xr-x 77 root wheel 119008 4 Aug 12:31 /usr/bin/pip3
-rwxr-xr-x 77 root wheel 119008 4 Aug 12:31 /usr/bin/python3I was having trouble understanding the scenario you seem to have in mind because it never occurred to me that someone would try to run Python without doing an install from python.org as I explicitly recommend.
If someone does install a version of Python from python.org, it puts the bin folder for that version first on PATH via this line in .zprofile:
PATH="/Library/Frameworks/Python.framework/Versions/3.11/bin:${PATH}" export PATH
It also puts a symlink for python3 and pip3 into the `/usr/local/bin` folder that comes ahead of `/usr/bin`.
So if the user runs
`pip3 install ...`
it will not find the version in `/usr/bin` because there will be two other directories ahead of it on the path that have an instance of `pip3`.
As an aside, if they do the improbable and run something like
`/usr/bin/pip3 install ...`
or if they do not have any Python from python.org installed and run
`pip3 install`
what they will end up with is a user-install that puts libraries in
`~/Library/`
because at least on an Apple Silicon mac, pip can't write to `/usr/bin`. This fallback to a user-install is confusing to people who encounter it, but it is very different from "installing to the system python."
To summarize, it is exceedingly unlikely that a student who is not comfortable running commands from the terminal is going make the mistake you seem to be worried about and end up with libraries in the user-install location:
1. They do not install an official Python even though that is exactly what I recommend.
2. They do install XCode or XCode Command Line tools, then try to use `pip3 install ...`
> You will not be forced to work inside something known as a “virtual environment.”
Oof, this terrible advice cancels out an otherwise reasonable post. Beginners who don't know what they're doing are the last people who should be `pip install -r requirements.txt`-ing into the system Python the way this article is recommending. That's not only going to make working on multiple projects nearly impossible (especially for the kind of beginning students who get recommended Anaconda, which are almost always the Data Science-y crowd using NumPy, Pandas, Scikit, etc, which are notoriously finicky with version conflicts), but it stands a good chance of breaking other Python-based system utilities in completely opaque ways. This sort of advice can fubar a naive user's entire workflow.
I know virtualenvs suck to explain to people, but in my opinion it needs to be done before you ever tell them about `pip install`.