I really don't get the fuss about the global dependency management though, maybe I would change my mind if Python shipedp a great implementation of it. But I feel like the problem is already solvable in multiple ways with containers, VMs, or virtualenvs and I don't think yet another abstraction to separate environments would add much value to my day to day workflow building python apps.
And yet, I hear so many more complaints about Python pip and I really don’t understand the disconnect. Perhaps dislike of pip is actually triggered by usability issues? And then people look for other reasons to explain their dislike?
As mentioned elsewhere in this thread, resolving this dependency issue would require a change in the Python language itself.
If you're only working with the standard library, boto3, twisted and redis - you're unlikely to have issues. You get into big issues, when you get to more obscure libraries... or libraries that are C bindings.
There are per python binary, per user, per virtualenv installation (per project or per whatever you like) that make the conflict less likely.
Sometime packages "vendor" their dependencies e.g., there is `pip._vendor.requests` (thus you may have different `requests` versions in the same environment).
There were setuptools' multi-version installs https://packaging.python.org/guides/multi-version-installs/ (I don't remember using it explicitly ever -- no need)
That's not pip's fault, that's Python's fault. Python's module system has no concept of versioning, so there can only ever be one copy of a module that has a given name.
And this is an interpreter detail that is exposed through the language itself, so it can't be fixed without causing severe pain.
That's a bug, not a feature. It enables sloppy development and the disasters like on NPM
> Now whether or not it's a good idea to support this kind of dependency stuff can be controversial. In practice though clearly it does cause problems the larger your codebase gets as you're more likely to have some nested dependency chain that's time-consuming to upgrade so you'd rather move faster than make sure you're only running 1 version of the dependency.
Consider the view that some times it can be sloppy & other times it's not & it's impossible to distinguish between the two in an automated fashion.
If a package is just using a particular library internally, I don’t see why the package manager should prevent using it with another library that depends on a different version.
I do. The main reason for Linux distributions to exists is to provide a development and running environment where:
- API/ABIs do not change for the whole lifetime of the distribution. No new features, no new bugs, no new vulnerabilities, so that your production code can run reliably for 5+ years.
- Vulnerabilities are fixed with minimally invasive patches.
- Vulnerabilities are fixed in reasonable times even if the upstream development stopped. Patches are well tested against the set of packages in the distribution.
You simply cannot have these 3 features together if a distribution ships 10 different version of each library.
It's already a ton of work to maintain packages in stable distributions.
Some of this has been mitigated with virtualenv but having a project express it's packages & have that automatically reflected in the environment.
Finally, Cargo to my knowledge actually lets multiple dependencies exist (even within the same project!!!) so that you can have a dependency like:
< my awesome project> -------- dep3 >= 3.0 \ dep2 -- dep3 >= 2.0That's not possible if you don't have the right language hooks because module resolution needs to be aware of the version of the library (i.e. when you go `import numpy`, it actually needs to be aware of the package it's being imported from to resolve that correctly).
Now whether or not it's a good idea to support this kind of dependency stuff can be controversial. In practice though clearly it does cause problems the larger your codebase gets as you're more likely to have some nested dependency chain that's time-consuming to upgrade so you'd rather move faster than make sure you're only running 1 version of the dependency.