Well, I would not want to copy a virtualenv from one machine onto another - that approach sounds like it's fraught with trouble.

I think the sane way to do it is to package your app using setuptools (and list dependencies) and then use pip install to install it in a new virtualenv on the production machines.

Here's how I do it on the project I'm currently on:

https://github.com/InsolvencyService/rps-alpha/blob/master/s...

Ruby gets it right, too. You can have a very simple gem setup if you want it - set GEM_HOME and away you go. I'm sure it's possible to get the same simplicity with Python.

The issue I see is that the default tools people turn to hide this simplicity behind a shiny interface, and encourage people to think of the management tools as magic black boxes. The tools then become the only interface people know, and they're inevitably more complicated and fragile than the underlying mechanism.

Python predates Java and .NET (never mind NPM), and was originally more systems-oriented; the idea is that you install libraries globally, using your system package manager, just like with C libraries. pip/virtualenv had to be retrofitted on afterwards.

because (and correct me if I am wrong) Jars (and certainly Wars) contain other bits than the part you wrote.

let's say you are packaging up a java and a python "program". Both print Hello world to the stdout, but use a third party spell checking package. Both the venv and the jar will contain that third party package

All python needs to go from venv to jar is a tarball process.

that is roughly were I see us going anyway - ps anyone with good understanding of jar files please jump in - I would love to map out the python parts against the jar parts (ie pip install is what part of the java setup process?)

Python also understands zip format for modules. So you run

zip -r mymodule.zip mymodule/

and you have the same portability as you have with jar. For bonus points, you can append that zip to a copy of the python executable (either python.exe or whatever binary your OS uses) and it is available as a module for that instance of python. This is one way standalone Python program executables are distributed.

(this was only tangentially related, but I think it's cool. I may also be wrong)

> but use a third party spell checking package. Both the venv and the jar will contain that third party package

Jar files do not support including third party libraries. You simply cannot put a jar in a jar. Eclipse/maven etc can explode the dependancies class files inside of the jar, but people rarely do that for anything other then distributing binaries of client apps (licensing becomes a massive pain btw).

war/ear files on the other hand does support including dependant jar files.

I may have mis-spoke (could not re-read thanks to my antiprocrast) : I am thinking of a package format, probably more similar to .deb than .jar, that will drop the code I have written, plus the third party packages, plus the system level packages, ie all the dependancies that can get compiled)

The idea is to divide the Operating system from the application. There is never a good divide point but at some point we can say the chef/puppet/salt infrastructure will give us the operating system, and the big binary blob (.deb) will give us everything else.

Throw in configuration as a seperate function and I am going to have to lie down in a dark room.

>Why? Why can't I just have the pip'ed libraries in a subdir somewhere and tell Python to look there?

Because that ties you to a single system wide python. Why would you want that?

> NPM gets this right

https://news.ycombinator.com/item?id=6859810

Yeahyeah, NPM and its community get plenty other things wrong, but not the particular flavour of dependency hell that we're discussing here. It's all neatly there, in a local node_modules subdirectory, and assuming the modules have been properly maintained, it just works.

You have to do actual effort to install an npm module globally (namely, provide a switch). The default does the sane thing.

>You have to do actual effort to install an npm module globally (namely, provide a switch). The default does the sane thing.

No it doesn't? You have to manually create a node_modules dir in the current directory because otherwise it'll scan up your tree for any parent dir containing node_modules, which is pretty much never what you want.

The npm registry can be replicated as CouchDB is the underlying data store.

http://clock.co.uk/tech-blogs/how-to-create-a-private-npmjs-...

python actually has reasonably good support for this via a single environment variable (PYTHONENV or PYTHONHOME, I think, it's been a while since I looked at this). The problem is that it overrides all the default search/path locations, and thus is missing the complete stdlib unless you copy everything into it. There's also the matter of binary modules that can not be shared between some interpreter versions (which feels like it begat Ubuntu's questionable python-dist symlink nightmare).

Nope, PYTHONPATH just prepends to sys.path. Does exactly what you say it doesn't do. It's virtualenv that can drop the system, thus necessitating a local copy.

I make no mention of PYTHONPATH.

> NPM gets this right.

Until you want to actually deploy to production -- then good luck, write your own tools.