Re:Python's New Package Landscape
by Bogdanp on Thursday October 18, 2018 @08:51AM
(#18247854)
It takes about 2 minutes (feels like 5!) on my 2016 MBP to install 102 dependencies. Doing that in Docker takes about 1.5x the time. I haven't seen it take 20-30 minutes, but 2-3 minutes is still obscenely slow in my view.
[ Parent
| on HN ]
Re:Python's New Package Landscape
by scrollaway on Thursday October 18, 2018 @09:07AM
(#18247970)
A lot of this time may be spent on downloading the dependencies to the cache. If you're doing it in docker, you likely don't have a persistent cache. I've hit this issue before. https://github.com/pypa/pipenv/issues/1785
If you configure the cache properly you might solve it, but yeah it's kinda dumb it has to do that just for locking.
There is no reason for Docker to be slower, it must be some kind of configuration issue. Containers are basically just processes and there is virtually no difference in execution times.
Re:Python's New Package Landscape
by scrollaway on Thursday October 18, 2018 @03:04PM
(#18251171)
As I said in my other reply, Docker is more likely to have ephemeral storage for the cache. So every single lock it'll re-download the package. Whereas locally, you're likely to still have the packages cached.
This can make a difference of tens of minutes for some packages which have a 1 gigabyte (!!!) download.