News for hackers. Stuff that matters.
Re:Python Virtual Environments – a Primer by crucialfelix on Monday March 28, 2016 @11:26AM (#11374804)
I've switched to using conda for managing all of my local python projects and tools. Its an easier, more flexible interface and it can switch both the virtualenv and the python version. So no problem going back and forth from py3 to 2

Just install miniconda (without all the data science packages) and then use pip as normal

http://conda.pydata.org/docs/using/envs.html

Autoswitching when you enter a directory:

https://github.com/chdoig/conda-auto-env

Re:Python Virtual Environments – a Primer by p4wnc6 on Monday March 28, 2016 @12:22PM (#11375181)
Most often you won't need to use pip, and `conda install ...` is preferable (for example, pretty much everything on PyPI is mirrored to anaconda.org, and if you ask them to support a well-known package that's not already available, it will be available extremely quickly, and chances are it's already available on some other anaconda.org channel anyway).

You can use pip from within conda as well if you want to, and you can refer to environment-specific pips directly, such as `~/anaconda/envs/my_env_name/bin/pip install ...` to install for the environment my_env_name.

conda can work with pip-styled requirements.txt files as well, and it's easy to either upload your own packages to anaconda.org, or create the conda build metadata (metadata.yaml) with instructions for how to get the package from PyPI if you prefer to only upload to PyPI.

There is an abundance of well-written tutorials and documentation for these things.

Re:Python Virtual Environments – a Primer by epaulson on Monday March 28, 2016 @12:47PM (#11375401)
It'd be really great if conda knew how to fall back to pip, or if the conda skeleton -recursive / conda build / conda install cycle could be collapsed into one command
Re:Python Virtual Environments – a Primer by p4wnc6 on Monday March 28, 2016 @02:22PM (#11376179)
I'm not sure falling back to pip is a good default. You might rather be alerted to a package's absence on anaconda.org than to have it silently installed elsewhere, for example for use cases when you totally want zero pip installed packages.

But I agree there's a lot of opportunity for improvement around the configurability of this sort of thing.

Re:Python Virtual Environments – a Primer by darkarmani on Monday March 28, 2016 @04:40PM (#11377215)
It used to fall back to pip, but it was taken out. It's confusing if you typo the package name and then it tries to find it on PyPI and can't.
Re:Python Virtual Environments – a Primer by crucialfelix on Monday March 28, 2016 @01:56PM (#11375971)
That's good to hear. Why is using conda install preferable ? Is it new infrastructure or something that you think might overtake PyPI ?
Re:Python Virtual Environments – a Primer by p4wnc6 on Monday March 28, 2016 @02:30PM (#11376227)
While conda is newer than virtualenv, pip, and easy_install, it's not really "new." Conda also is orthogonal to the source repository. As I mentioned, you can easily let conda manage the environments of your pip-installed packages.

One reason anaconda.org is better is that it's also Python agnostic. There is a main channel for R and R packages, so you can actually manage R environments the same way and R dependencies in a Python project. This could conceivably grow to include many things that pip alone can't help you with.

Having built packages both by using the necessary extra setting in a setup.py file, coupled with `python setup.py register` and the PyPI upload commands, and also via the meta.yaml and the anaconda.org upload commands, the latter seems more sensible to me.

Putting info into setup.py is a bad thing generally, especially if you do real work in setup.py, like with a complicated Cython package. Placing that info into meta.yaml offers better separation between data and code, and is remarkably easy to use in a cross-platform way with just `conda build`.

There's a lot more related to configurability and advanced options, but the above examples have been enough for me to fully use conda and to support other packaging as secondary when necessary.

Re:Python Virtual Environments – a Primer by StavrosK on Monday March 28, 2016 @03:47PM (#11376805)
Hmm, can someone mention the advantages of conda? Right now I use virtualenv with autoswitching, and I don't know what conda provides that's better. You mention switching the Python version, but virtualenv does that seamlessly as well, so I never had a problem.
Re:Python Virtual Environments – a Primer by crucialfelix on Tuesday March 29, 2016 @06:07AM (#11380492)
yes, virtualenv switches to whatever python you used to create that virtualenv. but where did that python come from ?

conda installs pythons as well. like pyenv, but I lost a bunch of time trying to get that to work correctly.

everytime I used virtualenv I had to go look up things and many things still didn't work with the same ease as nvm rvm etc. There's virtualenv, virtualenvwrapper (to make it somewhat usable ? I forget); pyenv; pyenv-virtualenv (what?)

conda has always worked flawlessly for me without fussing.

Re:Python Virtual Environments – a Primer by StavrosK on Tuesday March 29, 2016 @09:51AM (#11381433)
Oh, it installs its own Python version? That's great, I needed 3.2 the other day but only had 3.4. I'll give conda a try, thanks!
Re:Python Virtual Environments – a Primer by japhyr on Monday March 28, 2016 @12:09PM (#11375078)
How well do Django-based projects work with conda?
Re:Python Virtual Environments – a Primer by crucialfelix on Monday March 28, 2016 @12:12PM (#11375110)
I'm running that in Vagrant anyway, but it would work just fine. Conda is just a well done wrapper for switching python versions and virtualenvironments.

You can also set ENVIRONMENT variables for each so that could be useful for specific django projects.

Re:Python Virtual Environments – a Primer by pekk on Monday March 28, 2016 @02:21PM (#11376169)
What do you do about all the missing packages?
Re:Python Virtual Environments – a Primer by beagle3 on Monday March 28, 2016 @03:18PM (#11376605)
pip install <package>

Works perfectly well inside conda, and properly tracked. Packages don't have to come from anaconda/binstar.

Re:Python Virtual Environments – a Primer by davexunit on Monday March 28, 2016 @12:05PM (#11375055)
I currently use GNU Guix to make virtual environments [0] (sometimes using containers) for software written in any language, which I find much nicer than a Python specific solution.

[0] http://www.gnu.org/software/guix/manual/html_node/Invoking-g...

Re:Python Virtual Environments – a Primer by berdario on Monday March 28, 2016 @01:18PM (#11375652)
I also think that Nix/Guix are the way forward... Guix really needs an installer/Linux package, though

Just yesterday I added a few tasks to the ansible playbook for my workstation to automate its installation:

https://github.com/berdario/dotfiles/commit/4e92cdb00dbe135f...

Small disclaimer: I'm also the author of Pew ( https://github.com/berdario/pew ), an alternative to virtualenvwrapper... and I'm working on a small program to try to automate further what pypi2nix/guix-import do to automatically package Python stuff for Nix/Guix... Until that'll be really easy, I think that whipping up a new pew virtualenv to try something out quickly is still the easiest thing (especially for people not used to Nix... I know: otherwise you can also use mutable virtualenvs on top of Nix)

Re:Python Virtual Environments – a Primer by actionscripted on Monday March 28, 2016 @01:05PM (#11375548)
We've moved to Docker Compose for local Python (Django) development.

Oftentimes when you need package management (pip) and multiple environments (virtualenv) you likely also have several services necessary for your projects (Python, Postgres, Nginx, Celery, et al.) and it can be easier for your team to run `docker-compose up` and have the whole stack running in lieu of setting up services manually and then initializing a virtual environment.

I much prefer taking the time to setup Docker Compose even for a simple project. I don't have to initialize a virtual environment, install packages, make sure the DB is running, sync with the DB, run the web server, etc. All of that work becomes automatic and I can focus on development instead sysadmin stuff.

Re:Python Virtual Environments – a Primer by ris on Monday March 28, 2016 @01:47PM (#11375885)
Frankly I just use Nix, which is a little like having virtualenvs for your entire distribution and as such can handle non-python dependencies properly.
Re:Python Virtual Environments – a Primer by pekk on Monday March 28, 2016 @02:23PM (#11376184)
Virtualenv isn't supposed to handle non-Python dependencies.

Why use Nix instead of containers or VMs?

Re:Python Virtual Environments – a Primer by ris on Monday March 28, 2016 @07:02PM (#11378171)
"Virtualenv isn't supposed to handle non-Python dependencies."

Making it kinda useless for anything like numpy or scipy.

"Why use Nix instead of containers or VMs?"

Well, why use virtualenvs instead of containers or VMs?

Because it's lighter weight and more straightforward. And doesn't require you to swap out your whole working environment every time you want a slightly different package setup.

Containers and VMs are a bad excuse for "I don't have a hope of getting the dependencies of my setup under control so I'm just going to ball them all up into this great big binary image."

Nix packages are incredibly flexible and composable in comparison.

Re:Python Virtual Environments – a Primer by njharman on Monday March 28, 2016 @11:33AM (#11374852)
I've been very happy with vex https://pypi.python.org/pypi/vex
Re:Python Virtual Environments – a Primer by jaxondu on Monday March 28, 2016 @02:25PM (#11376202)
I don't need the extra in virtualenvwrapper, just use the build-in venv module in Python 3:

$ python3 -m venv myvenv $ source myvenv/bin/activate

Re:Python Virtual Environments – a Primer by sk3w on Monday March 28, 2016 @11:01AM (#11374629)
Note that pyvenv != pyenv, and the linked SO article appears relevant to the latter, not the former. Naming is hard.
Re:Python Virtual Environments – a Primer by recursive on Monday March 28, 2016 @05:01PM (#11377366)
I was puzzling over that exact thing for maybe 15 minutes. I've never used either one. Neither is on my path currently, even though I just installed python 3.5 with that option ticked. I can find pyvenv.py in my file system. Perhaps I'm expected to add it to my system path manually? Maybe I'll figure this out another day.
Re:Python Virtual Environments – a Primer by ubernostrum on Monday March 28, 2016 @02:55PM (#11376434)
Well, pyenv does have rather good virtualenv integration, and in fact I use that to make it easy to spin up a virtualenv with whatever Python version I happen to need at the moment.
Re:Python Virtual Environments – a Primer by STRiDEX on Monday March 28, 2016 @12:07PM (#11375067)
Using virtualenvwrapper and the plugin included with oh my zsh virtualenviroments are easy, but it is good to know what's happening.
Re:Python Virtual Environments – a Primer by beagle3 on Monday March 28, 2016 @11:18AM (#11374759)
I do not have a lot of experience, but I do have some, and in my experience miniconda is superior to virtual env.
Re:Python Virtual Environments – a Primer by lqdc13 on Monday March 28, 2016 @11:20AM (#11374766)
Some packages don't install correctly with *conda.
Re:Python Virtual Environments – a Primer by crucialfelix on Monday March 28, 2016 @11:27AM (#11374809)
you don't have to use conda to install. use pip as normal.
Re:Python Virtual Environments – a Primer by lqdc13 on Monday March 28, 2016 @11:55AM (#11374985)
Just looked at it again. Seems like it's better than before. Will give it a try.