equinoxe

The gemfile.lock (and I assume the node equivalent) are reactive things in the sense that they're generated for the final configuration (much like pip freeze).

I'm not really against of tracking dependencies recursively, but not for the final purpose of building a grand requirements file (which people will end up using and tweaking). I'm more interested if there are conflicts between different 3rd party libs (e.g. if one needs package==1.0.0 and the other needs package=1.2.0) and if they're not necessarily backwards compatible (e.g. v1.4 vs v1.6 of django).

Now, if you want to change the version of a 3rd party which, in turn, has altered dependencies, you'd need to clean up the venv anyway to be on the safe side. Currently I end up just rebuilding the venv and performing a pip install. On distribution, you could build an equivalent of a gemfile.lock via pip freeze (again). Hence, no real benefit of a pip-compile. Still, a pip diff (e.g. between 2 requirements.txt, tracking recursively dependencies) would be of more value imho, particularly if the devs would agree on a pattern/best practice.

There's also the matter of adding yet another file (requirements.in) to the setup framework (setup.py, metafiles...) and somewhat changing the meaning of requirements.txt.

I'd be more for a "pip diff" in that case, to show me what are the differences between the current venv and the changed layout.

I don't really like the idea of consolidating dependencies locally because you'll end up with a big compiled requirements.txt which then you have to maintain yourself. IMHO it'll be a drag when you'll update a 3rd party dependency which has its own changed dependencies...

Usually people code in one language (python in this case). learning another language for package management would require extra effort. Also, setting up the package is something you do once per project and forget about it (ok, you update it but far less frequent than your real code).

To be more exact, how many times have you customised your setup.py file? edited its dependencies and so on..? Imagine doing that in a totally different system (like debian packaging).

You can even now manage your dependencies and install scripts with something like automake or a homemade solution. Not really portable but if you don't care about your code being a module part of a bigger system...

The sad part is that I end up having multiple requirements files anyway to match the different environments (test, production, development) -- see the django applications for this. IMHO (again) we'd need some sort of structured requirements for different environments.

If I install an app/package for development, then I'd need development dependencies (like test-related packages etc.). I'm not supposed to know/care what the third party code needs for testing; just when I start testing the relevant code should "just work". Also, if I'm in production the testing-related packages should not pollute my environment...

I personally don't feel that pip-compile is that useful. All you can achieve with it can be achieved with a "pip install" + freeze in a clean venv. OK, error detection may have some use, but you still end with almost the same effort (packages are downloaded, parsing happens and you get an error).

IMHO we'd first need to standardize how dependencies are managed (requirements.txt, setup.py ...)and then build a solution off that. if you have requirements.txt-based setup, is easy to e.g. recursively download/parse to get the full list of dependencies.

The requirements.in file reminds me of automake.