Re:Show HN: Update your requirements.txt file with Pur in Py...
by tedmiston on Monday May 09, 2016 @01:03PM
(#11661332)
Why would you want to update requirements.txt before testing against the new package version first?
My dependency upgrade workflow for a Python web app is usually:
run pip list --outdated
upgrade one package
run tests
if success
pip freeze to update this package (and its dependencies)
commit
else
investigate or revert and make a note for later (if many breaking changes, etc.)
Perhaps this could be useful if you're using tox to test against many versions of Python, e.g. for a library.
(Or if you only freeze top-level dependencies, even though that goes against best practices.)
Re:Show HN: Update your requirements.txt file with Pur in Py...
by welder on Monday May 09, 2016 @01:08PM
(#11661366)
Some of my projects have double digits dependencies in their requirements.txt file so upgrading one at a time, even with pip list --outdated, would take too long.
I prefer to upgrade everything and fix things that break. I've always been able to easily figure out which dependency was causing the exception after upgrading all packages, so no point in upgrading them individually for me.
Also, I don't manually update my packages. My build scripts do that for me from my requirements.txt file, so it makes sense to update requirements.txt first.
Re:Show HN: Update your requirements.txt file with Pur in Py...
by ubernostrum on Monday May 09, 2016 @07:19PM
(#11664046)
Some of my projects have double digits dependencies in their requirements.txt file so upgrading one at a time, even with pip list --outdated, would take too long.
The only solution I know of to this is to "unroll" all those dependencies into your own requirements.txt, and manage them the same way.
For some projects I work with this is actually the only option, since I have to use requirements files with hashes and those are all-or-nothing -- even a single dependency-of-a-dependency that doesn't specify a hash will break the install, so I have to track down all of them and get a hashed entry into my requirements file.
Re:Show HN: Update your requirements.txt file with Pur in Py...
by tedmiston on Monday May 09, 2016 @01:35PM
(#11661566)
My main project has ~30. But still I prefer to go one at a time, especially with a major version bump, for the sake of understanding exactly what breaks. Perhaps it's just a personal preference.
Re:Show HN: Update your requirements.txt file with Pur in Py...
by orf on Monday May 09, 2016 @02:38PM
(#11662056)
Testing packages 1 by 1 finds any bugs and the exact package that caused it immediately, but you could just upgrade all of them and see if it fails. If it does a git-bisect could find the exact package that causes it.
Seems optimizing for the happy path (nothing breaks) could save some time.
Re:Show HN: Update your requirements.txt file with Pur in Py...
by emidln on Monday May 09, 2016 @03:18PM
(#11662374)
The tooling in Clojure land actually automated this (upgrade one dep, run tests, stop and back out if broken, continue otherwise). It would be neat seeing an equivalent of the lein-ancient plugin for Python, because this really is a PITA.
Re:Show HN: Update your requirements.txt file with Pur in Py...
by welder on Monday May 09, 2016 @05:45PM
(#11663478)
I explained in more detail in another comment here, but I don't directly install requirements so install first solutions like pip-review and pip list --outdated don't work for me.
Re:Show HN: Update your requirements.txt file with Pur in Py...
by welder on Monday May 09, 2016 @01:26PM
(#11661503)
pip-tools and pip-review just look at your installed dependencies and upgrade them, they don't update your requirements.txt file.
Upgrading packages doesn't work for me because I need my build scripts to do the upgrade. For example, when installing some packages which have C bindings they fail unless certain flags are set correctly.
Requires.io is cool, I personally use Gemnasium [1] for the same thing. However, both tools don't update your requirements.txt file for you, they just notify you when you need to update it.
Re:Show HN: Update your requirements.txt file with Pur in Py...
by nicois on Monday May 09, 2016 @05:37PM
(#11663419)
My approach:
keep my "raw" requirements.\.txt files in a ".requirements" directory in the project root. The packages listed here mainly are bare, without a version qualifier.
I run the script at https://gist.github.com/nicois/b49deb1f92e9f504cd93715a78440...
This creates matching requirements. files in the project root, with specific python package versions for everything.
Both sets of files are checked into the repository.
Periodically, I re-run the gist, which updates the requirements in the repo.
IMO this is the best of both worlds: in the "bare" requirements I only put what I want, and the derived requirements file is a fully-defined list of what I need.
This means each commit in my repo is not susceptible to third-party changes: when my tests pass on a given commit, it will pass every time in the future too. I can decide when I want to refresh the requirements, and it's a simple one-liner.
Re:Show HN: Update your requirements.txt file with Pur in Py...
by michaelmcmillan on Monday May 09, 2016 @02:46PM
(#11662116)
There is a free service that does this for you. It automatically sends a Pull-Request whenever one of your dependencies are outdated. This means that you can easily check that all your tests pass before merging it.
Re:Show HN: Update your requirements.txt file with Pur in Py...
by michaelmcmillan on Tuesday May 10, 2016 @02:25AM
(#11665483)
Yes, it is a new service, but I don't see how that holds up as general argument towards not using it. Your second point is false, it supports private repositories.
My dependency upgrade workflow for a Python web app is usually:
Perhaps this could be useful if you're using tox to test against many versions of Python, e.g. for a library.(Or if you only freeze top-level dependencies, even though that goes against best practices.)
I prefer to upgrade everything and fix things that break. I've always been able to easily figure out which dependency was causing the exception after upgrading all packages, so no point in upgrading them individually for me.
Also, I don't manually update my packages. My build scripts do that for me from my requirements.txt file, so it makes sense to update requirements.txt first.
The only solution I know of to this is to "unroll" all those dependencies into your own requirements.txt, and manage them the same way.
For some projects I work with this is actually the only option, since I have to use requirements files with hashes and those are all-or-nothing -- even a single dependency-of-a-dependency that doesn't specify a hash will break the install, so I have to track down all of them and get a hashed entry into my requirements file.
Seems optimizing for the happy path (nothing breaks) could save some time.
Another similar easy option is pip-review.
https://github.com/jgonggrijp/pip-review
If you want to automate updating your requirements file, take a look at requires.io [3].
[1] https://github.com/nvie/pip-tools
[2] https://github.com/jgonggrijp/pip-review
[3] http://requires.io/
Upgrading packages doesn't work for me because I need my build scripts to do the upgrade. For example, when installing some packages which have C bindings they fail unless certain flags are set correctly.
Requires.io is cool, I personally use Gemnasium [1] for the same thing. However, both tools don't update your requirements.txt file for you, they just notify you when you need to update it.
[1] https://gemnasium.com/features
IMO this is the best of both worlds: in the "bare" requirements I only put what I want, and the derived requirements file is a fully-defined list of what I need.
This means each commit in my repo is not susceptible to third-party changes: when my tests pass on a given commit, it will pass every time in the future too. I can decide when I want to refresh the requirements, and it's a simple one-liner.
https://github.com/nvie/pip-tools
https://doppins.com
[1] http://www.kennethreitz.org/essays/a-better-pip-workflow
Tools like pur, pip-compile etc. should have been included a long time ago.