It may depend on you experience with each language. I have much more experience with Python than Go (almost none with Rust) and therefore I have much better time with python packaging tools (I don't remember a single issue, that I didn't find a satisfactory solution -- as much as possible in the packaging world with myriad conflicting use-cases)

For example, my experience with `cargo` (that I mostly use to install command-line utilities such as rg, fd, dust): it is great when it works as written in the instructions but sometimes it doesn't (running `cargo` may involve a lot of compiling -- in contrast to `pip` which can use wheels transparently and avoid compiling even for modules with C extensions -- I guess there might be a way to do something similar with `cargo` though not by default).

The need for a virtualenv has nothing to do with pip. Python only has “global” dependencies due to the way its import system works.

Ruby works the same way, but bundler dependency manager solves it anyway to give you per-project dependencies not just system-wide dependencies. (I believe other well-liked dependency managers like cargo are largely based on bundler's semantics).

Perhaps ruby was more "hackable" by bundler. (Bundler has now become part of ruby stdlib, but didn't start out that way, it definitely started hacking around the way the more fundamental stdlib 'rubygems' worked).

> Ruby works the same way

Kind of, if you ignore Rubygems, which is also part of stdlib at a lower level then bundler (and also, originally wasn't.)

> but bundler dependency manager solves it anyway to give you per-project dependencies not just system-wide dependencies.

It can do that because rubygems manages multiple installed versions of packages and allows per-project ("per call to require", potentially, IIRC) specification of which one to pull from the globally-installed versions (this was originally done by monkey patching require when rubygems was an add-on.) This lets bundler easily live on top of it providing per-project dependencies somewhat more smoothly than Rubygems does without requiring anything like a venv.

> Perhaps ruby was more "hackable" by bundler.

Ruby is ludicrously hackable, yes.

> (Bundler has now become part of ruby stdlib, but didn't start out that way, it definitely started hacking around the way the more fundamental stdlib 'rubygems' worked).

Rubygems also wasn't part of stdlib originally, and started out relying on hacking around the way Kernel#require works.

> It can do that because rubygems manages multiple installed versions of packages

Oh wow, the default python dependency manager only lets you have one version of each package installed system-wide?

Yeah, that is a limitation. As opposed to rubygems (the first dependency manager although as you say not originally built-in to ruby) which has system-wide install, but always let you have more than one version installed.

Without fixing that one way or another, there's no sensible way, true. virtualenv is certainly one way to fix it. I wonder if there would have been a more rubygems way to fix it.

Honestly I found the multiple versions approach more complex, confusing and more hacky. A virtualenv is just “node_modules” that also contains a Python executable.

It’s a directory - you delete and create them at will, fast, and don’t worry or care about the system Python. Having some crazy setup that patches “require” to handle concurrently installed package versions seems insane, especially if you cannot actually use them concurrently in the same Ruby process. So, segmenting them by project (aka virtualenv) seems like the best solution.

so they’ve just automated the virtual environment creation then. There’s nothing about pip that is global or not global. It unzips files from pypi into a directory. Python, not pip, doesn’t really support a “node_modules” style setup. We use virtual environments (venv/) which is somewhat similar.

Sort of. The equivalent of "virtualenv" would be more "rvm gemsets", which is what people did before bundler. Bundler is doing something different.

Bundler is not a "node_modules" style setup. It does not require dependencies to be in a local path (although they can be, the default is they live in a system-wide location, and this does not limit functionality). It also does not support more than one version of a dependency in the same execution environment (as node_modules does) -- that really would be impossible in ruby too.

It's possible something about python's design would make the bundler approach impossible, I don't know. But it's not "dependencies are installed globally" alone, as that's true of ruby too.

We would probably all benefit in understanding better how these things are handled in other environments. And I include myself here. I think ruby's bundler really set a new standard for best practices here, and many subsequent managers (like cargo) were heavily influenced by it, although many don't realize it. But meanwhile many don't even realize what they are missing or what's possible.

Like the basic idea of having a specification of top-level dependencies (including allowable ranges) separate from a "lockfile" of exact versions in use of ALL dependencies... is just so hugely useful I never want to do without it, and I think is compatible with just about any architecture, and yet somehow JS is still only slowly catching on to it.

Not quite true. Python import mechanics are quite hackable and controllable programmatically and externally via controlling the relevant PATH env variables. It's just that no one bothers with it and instead seems to rely on the set of global folder-path lookup mechanics that are standard.

But nothing stops you from masking global-libraries with local library versions (similar to node_modules). Why hasn't anyone done this you may ask? I don't know the answer to that.

There are a few reasons you can’t have multiple versions of the same module at the same time. Consider a simple enum class defined in a package: two versions now have two different enum objects which may not compare equally. Maybe a function from module A would return Enum.A and pass it to module B, which would then compare A.Enum.A to B.Enum.A, which fails. Super confusing.

So yes, Python’s import system is dynamic enough to do crazy things, but I don’t see how we can ever retrofit that into Python.

Regarding dependencies installed into a project-local directory (node_modules): that’s a virtual environment. Just more flexible.

What's wrong with virtual environments? The required tools come bundled with Python nowadays and are super easy to use.

Agreed 100%,in fact of the languages I work with regularly (Python, Java, C#, JavaScript, Go) Python has the simplest dependency management solution via virtualenv + pypi + pip. Not sure why every Python thread turns into a conversation about the pain of Python dep management... seems overblown.

The biggest piece that is missing is you have to go out of your way to get sane dependency management. I have only used JS and F# (same dependency management as C#) out of your mentions, and it's the official tools that enable the local dependency management with a single command ("npm install X" or "dotnet add package X").

If you're using python, you don't know to check out pyenv/etc until you have a huge mess on your computer due to pip's behavior.

Hm.... Let me see. The number of operations I have to perform in Maven vs Python for independent packages:

Maven Scala project - create skeleton, add libraries to POM, write app, run app

PIP Venv Python project - create venv, enable venv, create requirements file, write app, run pip to install dependencies(possibly install GCC and extra libraries), run app

(Oh... and god forbid that you forget to deactivate venv)

You're lying when you say that library management is easier in Python. It's just factually untrue.

You don't have to activate the environment. I never do; it's a strictly optional convenience (it you think it's convenient).

Instead, simply run the interpreter installed in the environment when you run your app, e.g. "./my_env/bin/python my_app.py", and things will just work. No activation required, no special mode, nothing to forget.

The part about requirements.txt and installing packages could also be simplified if you did it the other way around: install first and create the requirements file from that:

  $ python3 -m venv my_env
  $ my_env/bin/pip install some-dependency
  $ my_env/bin/pip freeze >requirements.txt
  $ my_env/bin/python3 my_app.py

There you go. Setup, install and run in four steps and zero modes.

That's literally one step more, than Maven.

That's before you get to package your app...

Most of the steps you're listing take only a few seconds. They're talking about the actual management, not how long it takes you to type . venv/bin/activate

A few seconds here and a few seconds there - it's death by a thousand papercuts.

There's no community consensus - that keeps Python from advancing to where it needs to be.

I said it once and I'll say it again - Python lacks mature tooling.

You can't have death by a thousand papercuts when there are exactly three delay-papercuts adjacent to a step that takes a significant amount of time.

There are contexts where little delays matter, and you didn't pick one of those.

Maven is far more complicated and burdensome when compared to virtualenv and pip... pom.xml, what a righteous mess of XML and overly specified nonsense.

This is just to run your app, to package it - it's a whole different headache in Python.

It's literally a `mvn jar` and that's it!

> so Python developers resort to hacks like virtual environments to get work done

It's really hard with many deps, it's why cabal (for instance) moved away from a global model.