News for hackers. Stuff that matters.
Re:Python's New Package Landscape by scrollaway on Thursday October 18, 2018 @08:47AM (#18247826)
> Pipenv lock can take 20-30 minutes on a small flask app (~18 dependencies)

Do you have scipy/numpy/keras or cython somewhere in the deps? pipenv lock is slow, but not 20-30 mins slow unless there's a very very large download and/or a long compilation somewhere in there.

Re:Python's New Package Landscape by whitehouse3 on Thursday October 18, 2018 @08:59AM (#18247907)
The web app in question depends on PANDAS + numpy so that's definitely part of the toolchain. It wasn't 20-30 minutes from day one. The lock time started fast and then ballooned. Other comments here saying 2-3 minutes per lock are consistent with my general experience.

This wasn't for complex pipenv operations either. A simple command: pipenv run python main.py took progressively longer to execute.

Re:Python's New Package Landscape by Bogdanp on Thursday October 18, 2018 @08:51AM (#18247854)
It takes about 2 minutes (feels like 5!) on my 2016 MBP to install 102 dependencies. Doing that in Docker takes about 1.5x the time. I haven't seen it take 20-30 minutes, but 2-3 minutes is still obscenely slow in my view.
Re:Python's New Package Landscape by scrollaway on Thursday October 18, 2018 @09:07AM (#18247970)
A lot of this time may be spent on downloading the dependencies to the cache. If you're doing it in docker, you likely don't have a persistent cache. I've hit this issue before. https://github.com/pypa/pipenv/issues/1785

If you configure the cache properly you might solve it, but yeah it's kinda dumb it has to do that just for locking.

Re:Python's New Package Landscape by Drdrdrq on Thursday October 18, 2018 @02:32PM (#18250867)
There is no reason for Docker to be slower, it must be some kind of configuration issue. Containers are basically just processes and there is virtually no difference in execution times.
Re:Python's New Package Landscape by scrollaway on Thursday October 18, 2018 @03:04PM (#18251171)
As I said in my other reply, Docker is more likely to have ephemeral storage for the cache. So every single lock it'll re-download the package. Whereas locally, you're likely to still have the packages cached.

This can make a difference of tens of minutes for some packages which have a 1 gigabyte (!!!) download.

Re:Python's New Package Landscape by drcongo on Thursday October 18, 2018 @09:22AM (#18248066)
I have a docker project with ~20 packages in the Pipfile, the lock step of a new `pipenv install` takes about 3 minutes.