Constraint solving is the status quo, and I am strongly opposed to it. Rather than saying "I want foo >= 1.0" in a package manager, you should have the build system test the environment for the features you need. This is what the Autotools and other build systems that have been around for decades do. This way, the project isn't tied to a single package manager and, in the case of free software, greatly eases the burden on package maintainers that want to add your software to their distribution.
Functional package management doesn't need constraint solving because every package in the system precisely describes itself: exactly which dependencies are needed for build-time and runtime (which precisely describe themselves, recursively, all the way down to libc), precisely which source code (tarball/directory/whatever + checksum), and the exact build script that turns that source code into a (hopefully bit-reproducible) binary. There's no constraint solving to determine what the dependency tree is, it's already been encoded in the package objects themselves. This is what any robust package manager should enable, but the only two that I know of that do this are Nix and GNU Guix.
>The problem then is now your language has a dependency on an outside package manager, one which is often OS specific. Most modern languages need to support a variety of OSes.
A programming language should not depend on any specific package manager. This is what build systems are for! More and more language communities conflate the two, but this is a huge mistake that we are paying for by making our software nearly impossible to reproduce from source code. [0] Everyone just uses pre-built binaries full of circular dependencies and reproducibility issues because things are so tangled that no one actually knows how to build anything from source anymore.
I think the value of language-specific package managers lies in providing an easy way to fetch pure (no native code) modules to faciliate simple code sharing and helping newcomers get bootstrapped quickly. However, for serious software development and deployment they are terrible and we desperately need better tools. I think functional package managers are the tools we need, as they have greatly simplified software building and deployment for me whilst also greatly increasing the reliability of the systems that use them thanks to transactional upgrades and rollbacks.
[0] http://www.vitavonni.de/blog/201503/2015031201-the-sad-state...
Is there a single non-language specific package manager which is cross platform, and supports a large percentage of packages for a given programming language?
Note that I'm not saying npm is excellent, but I'm not aware of a single successful language-independent package manager. If there are none, why?
One could argue that they aren't "successful" (in that they don't have all the packages you might want), but they are still quite new and are pretty different from traditional package managers. I believe that Nix/Guix are the future, but a future that will take some time to become widespread.
Not good enough, language package managers must work on Windows, with a regular command prompt.
Spack is build-from-source and takes a lot of inspiration from Nix and its descendants, but it adds support for things like swapping out compilers/MPI versions/etc., as well as a syntax for composing new builds of things on the fly That is, you can build something with a slightly different version of a dependency without ever editing a package file. Our users to be able to build a package with 6 different compilers and 3 different MPI library versions, and have their 18 versions coexist on the same system. This has been super helpful for testing production code at LLNL, because we have to run our stuff on other peoples' often very different clusters. For instance, at LANL, we prefer PGI compilers. At LLNL we tend to like Intel or Clang.
One reason languages tend to grow their own package managers is that there are all kinds of peculiarities about how different languages manage modules/extensions. Have you seen how many different ways there are to deploy Python modules? It's very hard to have multiple versions of things coexisting in the same Python env, and things like pkg_resources require editing client code [3]. Ew. Spack tries to address this by allowing different language packages to implement their own module activation/deactivation logic, so you can kind of have a language-specfic package manager for each install of the language. I'd rather have something more like Nix's profiles, though.
I wouldn't say that the project is huge yet, but we've had some interest from mostly other HPC sites. It would be cool if the project caught on in the "real" world, too.
[1] https://github.com/llnl/spack
[2] http://www.computer.org/csdl/proceedings/sc/2015/3723/00/280...
More importantly, part of the dependency graph, including compilers, is treated specially and essentialy considered outside of the scope of Spack. This seriously hinders reproducibility, as we tried to explain in https://hal.inria.fr/hal-01161771/en .
Ummmm... dpkg and yum? You know, the package managers that manage the software on everyone's production servers. Or do these not count because they don't work on Windows or something?
This is what we did with Dart's package manager[1]. It works like Bundler where it finds a single set of package versions that satisfy all of the constraints and there are no duplicate packages. It's a challenge because constraint solving is NP-complete, but it works very well in practice.
[1]: https://pub.dartlang.org/
> realize the inherent problems with limiting the scope of dependency management to a single language rather than the full dependency graph and switch to using functional package managers.
The problem then is now your language has a dependency on an outside package manager, one which is often OS specific. Most modern languages need to support a variety of OSes.