How do other people replicate your venv? How does production replicate your venv?

You pin all your direct dependencies in requirements.txt, run "pip freeze >requirements-frozen.txt" to get a list of the versions of all dependencies (direct and transitive), and then you ship that.

Replicating is just "virtualenv --python=python3 venv && . ./venv/bin/activate && pip install -r requirements-frozen.txt". Am I missing something here? Yes, to be absolutely sure, you need to have the same versions of python, pip, setuptools, gcc (for C extensions), C library dependencies, etc. on both systems that do the build, but that's the case for pretty much any build system out there if you want repeatable builds.

And if you you just don't feel like dealing with that, you can just use Docker (aka "put my machine in a tarball and ship it").

> You pin all your direct dependencies in requirements.txt, run "pip freeze >requirements-frozen.txt" to get a list of the versions of all dependencies (direct and transitive), and then you ship that.

What?! Seriously? That is not how it's supposed to work. Requirements.txt is already meant to be the frozen dependencies (like the output of pip freeze). Why change that? I much prefer how pip-tools does it with a new file, requirements.in that contains the unpinned direct dependencies. I thought poetry did the same thing but with pyproject.toml?

This is worse than how it’s done on most modern languages.

Different commands, but it's pretty much the same in similar languages: Ruby uses bundler configuration instead of virtualenv and gemfiles lock instead of requirements.txt. JS uses npm/yarn and lockfie. Go uses go.mod and does a lot of assumptions instead of virtualenv. (these days, it was worse)

Modern languages do pretty much the same, just have a nicer wrapper for it.

It's not "pretty much the same". Have a separation between Gemfile and Gemfile.lock is a huge improvement over just having requirements.txt. Having good, easy to use commands, that manages this stuff for your in a way that works well for the vast majority of use cases (i.e. bundler), is far better than the ill-defined, ad-hoc, mess that is requirements.txt management.

Gemfile-gemfile.lock is setup.py-requirements.txt.

And yes, as I said, a nice wrapper for management of it is missing in python. The main ideas/approach is the same though.

It’s really not the same. Gemfile lists the direct dependencies of your project with variable specificity of explicit versions. Gemfile.lock lists all dependencies after resolving your direct dependencies and locks them to specific versions. requirements.txt is some bastardization of both of these combined, and setup.py is neither of these it’s for distribution of modules/packages which Ruby has as well, but it’s not your gemfile.

edit: also, how do you manage dev dependencies with requirements.txt and setup.py?

You can put dev deps in extras in their own "dev" block. Similar to how gemfile uses groups. Use requirements-dev.txt for them. Requirements for is a full list of dependencies after resolution and their exact versions. (as produced by freeze) The main difference is that requirements is "as installed in current configuration", while gemfile.lock is "for any groups configuration".

Again, not as polished experience, but the model matches.

But it’s all ad hoc which is problematic. You’re solving it yourself, the tool isn’t solving it. It’s like saying that C is the same as python, you can just write your own garbage collector in C!

That's why we use Poetry.

yeah, that's what i tried to use for a while too. The problem is that so much of the ecosystem doesn't use it, and the python community fails to converge on a solution here the way other language communities do.

Sorry, I was a little terse — obviously you're right that you're not going to ship the virtualenv itself. As other people have said, the intent was that you create a requirements file (and/or maybe a setup.py if you need that functionality). What I meant is that between fiddliness and sluggishness of Poetry, and the marginal hassle of maintaining a requirements file and creating your own virtualenvs, I'm not sure Poetry is the clear winner at the moment.

Thanks. My main point is that bundler, maven (less so), and npm are all better than either venv/pip/reqruirements.txt or poetry. Additionally, it's also better that those language communities have mostly converged on those solutions.

Create setup.cfg (it is better than setup.py, since it is declarative) listing immediate dependencies, use requires.txt as a lock file (I prefer to use pip-compile from pip-tools).

Pipenv still works for me with everything version pinned.

That will probably be solved by the new pip resolver within the next few months.

Did that new resolver finally happen?

The pip SAT-based solver was supposed to land in 2017 or 2018 after a GSoC, and nothing came of it. Is it finally going to use a different heuristic than "#yolo" for picking which version of dependencies to install?

Ya, funded PSF grant https://wiki.python.org/psf/Pip2020DonorFundedRoadmap