News for hackers. Stuff that matters.
Re:Python's New Package Landscape by mlthoughts2018 on Thursday October 18, 2018 @09:01AM (#18247922)
I’m disappointed your post did not cover using conda. As the pipenv drama has rolled on, I’ve moved from viewing conda merely as the best user experience in Python environment & package management to instead viewing it as the only serious option for professional scientific computuing work (and quite possibly any professional Python work at all).
Re:Python's New Package Landscape by hcrisp on Thursday October 18, 2018 @09:16AM (#18248015)
Agree with you about using conda. And since no one has mentioned Jake Van der Plas' review of conda vs the alternatives, myths, etc., here it is:

https://jakevdp.github.io/blog/2016/08/25/conda-myths-and-mi...

Re:Python's New Package Landscape by rspeer on Thursday October 18, 2018 @11:44AM (#18249278)
I read that page and looked for the reason I don't use Conda (because I already have virtualenvs and I'm not prepared to burn them all down):

> Myth #5: conda doesn't work with virtualenv, so it's useless for my workflow

> Reality: You actually can install (some) conda packages within a virtualenv, but better is to use Conda's own environment manager: it is fully-compatible with pip and has several advantages over virtualenv.

> [...] the result seems to be fairly brittle – for example, trying to conda update python within the virtualenv fails in a very ungraceful and unrecoverable manner, seemingly related to the symlinks that underly virtualenv's architecture.

Doesn't sound like much of a myth then, if Conda's take on virtualenv is "you can technically do this, but everything will break ungracefully and unrecoverably, so please don't".

Re:Python's New Package Landscape by hcrisp on Thursday October 18, 2018 @12:44PM (#18249849)
He's not saying that you _should_ install conda within a virtualenv, but that some have tried with some success.

At the end, one of his conclusions is: "If you want to install Python packages within an Isolated environment, pip+virtualenv and conda+conda-env are mostly interchangeable". So don't change if you don't have to.

But he does give reasons why conda may be superior to virtualenv -- managing different version of Python, tracking non-Python dependencies, true isolation of environments, etc.

Re:Python's New Package Landscape by Keats on Thursday October 18, 2018 @10:30AM (#18248592)
I should probably write another post once I've tried conda a bit more. I've used it very recently for some numpy/pytorch environment and it was quite nice.
Re:Python's New Package Landscape by goerz on Thursday October 18, 2018 @06:53PM (#18252924)
Although conda is also getting slower and slower, and now routinely spends 5-20 minutes on dependency resolution even for trivial environments.
Re:Python's New Package Landscape by mlthoughts2018 on Thursday October 18, 2018 @07:08PM (#18253101)
This is deeply untrue for conda. Even very complex environments build in less than a minute. I can believe there are corner cases where conda is very slow, but claiming conda takes 5 minutes for trivial environments is flat out wrong. Perhaps it is issues with a firewall, VPN connection or something else, absolutely no chance that is from normally executing conda.
Re:Python's New Package Landscape by goerz on Thursday October 18, 2018 @07:53PM (#18253338)
Well, I experience it on a daily basis, and I'm not the only one! It's officially an open issue: https://github.com/conda/conda/issues/7239
Re:Python's New Package Landscape by mlthoughts2018 on Thursday October 18, 2018 @08:14PM (#18253425)
The slowest operation in the linked thread is still taking less than 2 minutes...

Edit: correction, there are two examples that take longer, one at 3.5 minutes, one around 8 minutes. I don’t think it changes any takeaways though.

Re:Python's New Package Landscape by goerz on Friday October 19, 2018 @12:11AM (#18254186)
The last one in that thread is 11 minutes. And, I just did `time conda create -n test -y anaconda pytest-cov pytest-xdist coverage sphinx_rtd_theme flake8` on my Macbook, and it clocked in at 36 minutes! Probably, this is affected by also having the condaforge channel active (which I need), but it's definitely not from some network/VPN issues. Everyone I work with has similar problems. So don't tell me there's "absolutely no chance that is from normally executing conda".
Re:Python's New Package Landscape by mlthoughts2018 on Friday October 19, 2018 @06:59AM (#18255743)
The same command for me, also running on a Macbook, took just over 5 minutes, using conda 4.5.9. I also repeated inside a Ubuntu docker container with only miniconda installed and got the same.

It’s funny to me that you would expect operations involving “conda install anaconda” to be fast though. 11 minutes seems perfectly fast for that and would be comparable to pip or anything else if doing that huge of a set of package installs.

That is not at all a “trivial environment” like you said previously.