I had my Python and pip installed from distribution packages. One day I needed to build something from source, but it needed Meson...

Ok installed Meson also from my distro packages! But no, the project required a newer Meson version.

Ok let's install Meson with pip! But no, it turns out pip packages themselves can require a minimum pip version!! Go figure.

So I couldn't build that program without first pip-installing Meson, and I couldn't pip-install Meson without first pip-installing a more modern version of pip itself.

Guess how well it worked when I upgraded pip. Spoiler: not a smooth way to discover that Python packaging is a joke.

Were you using a virtualenv? Or were you pip installing into your distro / system python, like a true savage?

I just knew the basics. So I was just following commands from stack overflow or similar. Silly me, thinking that it would work logically like other languages do! Later I ended up learning about all that stuff with venvs and whatnot, not to mention it's not even a unified solution but there are multiple alternatives, adding to the confusion... All this, no doubt, is a consequence of how terrible the Python packaging story is. Rust, Go, Ruby, Node, and more, I had used without remotely similar hassles.

You are right. It's not a good experience for new developers, especially given how long Python has been around.

They are so close, though! I think one lesson Python took is that it's better, even mandatory, to use venv and install stuff in isolated containment.

This is exactly what Node/NPM does: "npm instal" creates a new dir, "./node_modules" and installs stuff there, and next "npm" commands will transparently refer to that subdir. This is, in effect, the same thing as a Python's venv! to the point that in my scripts I always name them "python_modules", to give other unfamiliar devs a clue of what it is.

If "pip install" just did this transparently, without introducing the whole concept of venvs to the user, it would be a huge step forward. I cannot imagine how it would suck if NPM didn't make its node_module dir, and instead it forced you to first learn all the idiosyncrasies of module isolation.

Yeah, they should just define a new command line option (and perhaps a new command that defaults to this behavior, say vpip or vpython or something.) The virtualenvs take some getting use to. I've been using python for over 15 years so it's second nature.

Have you tried "poetry"?

Yes. Having tried it led me to write that comment.

I'm curious what problems you ran into with it?