This in fact was my intention when creating this feature in pip: setup.py has the abstract dependencies, requirements.txt is a recipe for building something exact. And I quickly found requirements-dev was useful for adding tools (some people have done that with setup.py extras, but the ergonomics on that never seemed great).

The one thing I wish was part of this is that there was a record of conflicts and purportedly successful combinations. That is, you distribute a package that says "I work with foo>=1.0" but you don't (and can't) know if the package works with foo 1.1 or 2.0. Semantic versioning feels like a fantasy that you could know, but you just can't. Understanding how versions work together is a discovery process, but we don't have a shared record of that discovery, and it doesn't belong to any one package version.

This sense that package releases and known good combinations are separate things developed at separate paces is also part of the motivation of requirements.txt, and maybe why people often moved away from setup.py

A pacakage/system that handled this would be great. Zope and Plone have the notion of "known good sets" -- and it's not really a pleasure to use (but much better than nothing). As far as I can tell, with Plone 5 - the recommended way to install Plone is from the unified installer - leaving known good sets, pinning and buildout to manage plugins:

http://docs.plone.org/manage/installing/installing_addons.ht...

For an example of what bootstrapping a full Plone 4 site via buildout entails, have a look at the (defunct) good-py project:

http://good-py.appspot.com/release/plone/4.2rc2

http://www.martinaspeli.net/articles/hello-good-py-a-known-g...

As long as one is able to manage to keep projects small, and in a virtual-env of it's own, managing "known good sets" (in buildout, or for pip) shouldn't really be too hard. But as projects grow, a real system for managing versions will be needed. As far as I know there are no good systems for this... yet. Ideally you'd want a list that people could update as they run into problems, so that if projectA runs fine with foo=1.0, and bar=1.1, maybe projectB discovers a bug in foo<=1.0.4rc5 and can update the requirement.

It's not a trivial thing (see also: All the package managers, apt, yum, etc).

Hi Ian, long time fan of your work.

I'm supporting both using some silly logic to pull in requirements.txt and supply that to setup.

https://github.com/russellballestrini/botoform

This. Using setup.py for specifying min versions (and occasionally max versions) is definitely the way to go.

However, manually maintaining a requirements file in addition to setup.py is quite tedious in the long run. It is much better to freeze requirements during the build process and use the generated requirements file for deployments. But, decent test coverage is key here.

I've bookmarked Donald Stufft's opinion on this for a couple years, which contrasts interestingly with Kenneth's and your own (and which makes a lot of sense): https://caremad.io/2013/07/setup-vs-requirement/

I like the concrete-vs-abstract, library-vs-application ideas there. Most opinions on this overlook the distinction.

I don't at all disagree with that, actually. I do consider that a part of setup.py, but it rarely matters in practice. The python ecosystem doesn't have similar programmatic api specs compared to, say, PHP where they get together and rigidly define these things: http://www.php-fig.org/psr/psr-7/

Not to say the PHP strategy is necessarily ideal (feels restrictive to me), rather just the comparison between the two. That's the sort of ecosystem that would make the abstract part matter more

No, no, Python very much does. PEP 0249 specifies DB API 2, PEP 0333 specifies WSGI, and PEP 3153 specifies the asyncio interface, for instance. If anything, I can think of more examples of it being done in the Python space than the PHP space.

This is easily the best option, barring how painful it is to write a setup.py.

Filling out setup.py and other package boilerplate used to annoy me as well but that all changed after I started using cookiecutter.

https://github.com/audreyr/cookiecutter

Thanks! I use cookiecutter and I still have to tweak things like MANIFEST.in and adding the package to the path while installing, for automatically-created entry points.

It's really not that bad with setuptools - as an added bonus it becomes easy to package your application up in an RPM.

setuptools? distutils you mean?

I much prefer fpm for creating Linux distro packaged from a python distribution, since it can create debs.

No, I mean setuptools - it does everything distutils does but better.

I don't trust random scripts to generate packages for me, writing an rpmspec or a debian control file is hardly a challenge and I'd encourage more people to take the 5 minutes to do so than relying on tools like FPM.

… or tools like setuptools?

Also, I assume you're talking about the bdist_rpm target that come from distutils.

No, I write a proper RPM spec file - only takes a few minutes.

So, what's the added bonus of using setuptools then?

setuptools is just less generally crufty than distutils, and things like entrypoints make writing installable console scripts feasible. Since it provides essentially the same interface the default rpmspec created by rpmdev-newspec works with it out of the box.