Yeah, that's essentially the right way (I use conda envs, but venvs are probably fine too). Have one env for running the Jupyter Lab/Notebook server (install jupyter, jupyter widgets, nbextensions, etc into this env) and then create other envs for your projects and register that kernel with jupyter [0]. You can also just install all of the jupyter ide cruft in each env if you want, but dependencies can get annoying.

[0] https://stackoverflow.com/questions/39604271/conda-environme....

This is the way. I use miniconda to create env for Jupyter and install only Jupyter and its dependency on that, then I just config it to point to all the other Python envs I use for my different projects. This is only one-time setup and it is absolutely worth it given how messy the ecosystem is.

> miniconda

Try micromamba, it will shave years off dependency resolution

I used to use micromamba, recently moved to mach-nix with nix. It's also really fast, but with the reproducibility of nix.

I just do a venv inside the venv so I can target several Python versions. I also use Jupiter for C#, Clojure, and Julia. I wish the Clojure integration was as good as the others I listed.

I’m to the point now where if anything other than venv/pip is required I won’t use it. Unfortunately there are many things that insist on conda.

Conda is a fucking nightmare. Luckily you can usually just use pip to get the same packages.

Why does “pip install gcc” keep failing then? ;)

Conda isn’t perfect but takes on a lot of problems that pip doesn’t deal with at all. Regular conda is really slow these days but you can use mamba instead or just configure conda to use the libmamba solver and it’s much nicer.

The folks at prefix.dev seem to be building some pretty cool drop in replacements for conda too.

One that you can’t is the Clojure kernel for Jupiter. I installed it from source. That’s no big deal, but they aren’t 1-to-1 with packages.

had a ton of trouble getting jupyterlab to target an arbitrary virtualenv

The trick is that you have to deactivate the virtual environment and then resource it after adding Jupyter to that virtual environment.

Most shells cache executable paths, so the path for jupyter will be the global path, not the one for your virtual environment. This is unfortunately not at all obvious and leads to very hard to track down bugs that seem to disappear and reappear if you aren't familiar with the issue.

I have a recipe here which always works: https://github.com/nlothian/m1_huggingface_diffusers_demo#se...

If you don't have requirements.txt then do this: `pip3 install jupyter` for that line, then `deactivate` and `source ./venv/bin/activate`.

I almost never rely on activating environments anymore. If it's in my project, I refer to it by its path relative to the project top level. If it's meant to be "global", I use Pipx, and if for some reason I can't, I use a script in ~/bin that uses the absolute path.

Right. I usually `pipx install jupyterlab` and register each project's env with `python -m ipykernel install --user --name MyProject --display-name "My Project"` as per the ipython docs: https://ipython.readthedocs.io/en/stable/install/kernel_inst...

Yeah this is the right way. But I don't think it's obvious from the getting started docs.

I bet a lot of people end up installing jupyter into each virtual environment instead.

This sort of works, as (I think) most people are only working on one or two notebooks at a time, and aren't using notebooks that relate to more than one virtual environment.

I do this, but it isn't well supported. Most Jupyterlab extensions expect you to `pip install` and that will handle both the frontend and backend components. The docs make basically no mention of the frontend/backend split. I end up having to install a bunch of frontend stuff in my project envs just to make sure my notebooks work.