> There aren't good boundaries between Jupyter's own Python environment, and that of your notebooks— if you have a dependency which conflicts with one of Jupyter's dependencies, then good luck.

It's cumbersome, and I'm not totally sure it's the correct way, but I remember getting around this by creating a virtualenv for my projects and then using that virtualenv's python as Jupyter's "kernel".

Yeah, that's essentially the right way (I use conda envs, but venvs are probably fine too). Have one env for running the Jupyter Lab/Notebook server (install jupyter, jupyter widgets, nbextensions, etc into this env) and then create other envs for your projects and register that kernel with jupyter [0]. You can also just install all of the jupyter ide cruft in each env if you want, but dependencies can get annoying.

[0] https://stackoverflow.com/questions/39604271/conda-environme....

This is the way. I use miniconda to create env for Jupyter and install only Jupyter and its dependency on that, then I just config it to point to all the other Python envs I use for my different projects. This is only one-time setup and it is absolutely worth it given how messy the ecosystem is.

> miniconda

Try micromamba, it will shave years off dependency resolution

I used to use micromamba, recently moved to mach-nix with nix. It's also really fast, but with the reproducibility of nix.

I just do a venv inside the venv so I can target several Python versions. I also use Jupiter for C#, Clojure, and Julia. I wish the Clojure integration was as good as the others I listed.

I’m to the point now where if anything other than venv/pip is required I won’t use it. Unfortunately there are many things that insist on conda.

Conda is a fucking nightmare. Luckily you can usually just use pip to get the same packages.

Why does “pip install gcc” keep failing then? ;)

Conda isn’t perfect but takes on a lot of problems that pip doesn’t deal with at all. Regular conda is really slow these days but you can use mamba instead or just configure conda to use the libmamba solver and it’s much nicer.

The folks at prefix.dev seem to be building some pretty cool drop in replacements for conda too.

One that you can’t is the Clojure kernel for Jupiter. I installed it from source. That’s no big deal, but they aren’t 1-to-1 with packages.

had a ton of trouble getting jupyterlab to target an arbitrary virtualenv

The trick is that you have to deactivate the virtual environment and then resource it after adding Jupyter to that virtual environment.

Most shells cache executable paths, so the path for jupyter will be the global path, not the one for your virtual environment. This is unfortunately not at all obvious and leads to very hard to track down bugs that seem to disappear and reappear if you aren't familiar with the issue.

I have a recipe here which always works: https://github.com/nlothian/m1_huggingface_diffusers_demo#se...

If you don't have requirements.txt then do this: `pip3 install jupyter` for that line, then `deactivate` and `source ./venv/bin/activate`.

I almost never rely on activating environments anymore. If it's in my project, I refer to it by its path relative to the project top level. If it's meant to be "global", I use Pipx, and if for some reason I can't, I use a script in ~/bin that uses the absolute path.

Right. I usually `pipx install jupyterlab` and register each project's env with `python -m ipykernel install --user --name MyProject --display-name "My Project"` as per the ipython docs: https://ipython.readthedocs.io/en/stable/install/kernel_inst...

Yeah this is the right way. But I don't think it's obvious from the getting started docs.

I bet a lot of people end up installing jupyter into each virtual environment instead.

This sort of works, as (I think) most people are only working on one or two notebooks at a time, and aren't using notebooks that relate to more than one virtual environment.

I do this, but it isn't well supported. Most Jupyterlab extensions expect you to `pip install` and that will handle both the frontend and backend components. The docs make basically no mention of the frontend/backend split. I end up having to install a bunch of frontend stuff in my project envs just to make sure my notebooks work.

At least for the last point I use conda with the nb_conda/nb_conda_kernels packages, which makes you able to switch kernels to different environments.

What do you use to collect telemetry data?

It's a mix of things, but the big one was ROS bag files for robots. So a lot of the conventional up-to-the-minute metrics workflows oriented around grafana/kibana don't really pan out when you're very interested in a five minute window that occurred three days ago, buried in a 1gb datafile.

I’m using Jupyter Book (https://jupyterbook.org/) to build all kinds of daily reports and it works really well. Individual notebooks can also be converted to HTML or pdf with nbconvert using command line or “download as” menu item.

+1 for Jupyter Book. I had to fight with it a bit for some things around LaTeX/PDF generation, but was very pleased with the result.

> The process to "promote" fragments of a notebook into being reusable functions seemed very high-friction: basically you're rewriting it as a normal Python package and then adding that to Jupyter's environment.

Don't get in that situation to begin with. Pop an `%load_ext` `%autoreload 2` at the top and just write functions in an imported .py file from the get go.

Ok, but let's also acknowledge that the mental model and levels of abstraction increased significantly, which presents a barrier to people without any prior software engineering experience, who are just getting into Python for "data science".

Well maybe we should expect people to learn the tools they use properly instead of constantly dumbing things down for the sake of convenience.

I’m not sure you should be able to just pop in and do “data science”

I don't think it's a huge leap to say your functions are stored in this .py file vs. in a cell above. the magic commands simply allow you to reload the functions automatically without needing to reimport / reload the kernel.

It is a huge gap when the notebook is documentation. Moving functions out either duplicates documentation or separates them. Notebooks are vastly superior for documentation. Extracting functions from notebooks into a py file isn't difficult (see nbdev for example)

Notebooks are good for demonstration, not documentation. They are bad for documentation because you need to put docstrings and comments in the code while desiring to put the same content in the markdown.

I typically don't find docstrings are particularly useful for documenting an analysis or understanding what a function is doing. Even analyses that are repeated with different data. When you are debugging or reviewing an analysis, the docstrings are the least of your concerns. Validity is far more important.

I would take a notebook that demonstrates what's happening to my data any day over a docstring that may or may not be correct. Particularly if I have to render it in yet a third system to juggle and manually keep in sync.

So we've gone from notebooks to... notebook + py + docstring... I'm sure we can think of another useless layer of indirection to bolt on.

> Particularly if I have to render it in yet a third system to juggle and manually keep in sync.

I'm not sure you know what a docstring is if this is your response.

If you are putting your docstrings in the py and they contain things like equations or figures or images, you have to render them to get the same effect that you would have just reviewing cells in the notebook. And as far as I know you can't even spread docstrings out over the body of a functions, the way you could if you broke the function up into cells. Docstrings are not literate programming.

> And as far as I know you can't even spread docstrings out over the body of a functions,

I really feel like you do not understand what a docstring is. This does not make any sense.

> the way you could if you broke the function up into cells

If your logic is broken up across cells then you cannot use it anywhere but in that notebook...

I realize you think the problem is that I don't understand what docstrings are, but it's more that you don't understand the sorts of documentation I'm talking about. docstrings are fine for a certain type of limited documentation that is formatted to prevent syntax errors. But they are extremely limited. Reading docstring/doxygen style documentation is the slowest and most error prone in my experience. Particularly when you are describing a complex calculation.

> If your logic is broken up across cells then you cannot use it anywhere but in that notebook...

You don't seem to understand how notebooks can be used and processed themselves. They are just data and there are libraries for loading and transforming them. You might check out pydev and papermill to get an idea about this (and maybe also literate programming in general).

I do all of my code in notebooks. If your docstring is slow to read, it is a bad docstring. To be clear, a docstring is the thing you get when you have a function, you don't know what it does, and you press shift tab to bring up the brief description of what it does in a pop up (in jupyter lab). It should be the quickest, least error prone form of documentation because it is available on demand and is supposed to be short.

Using notebooks as functions is a bad idea. I think papermill is trash.

Notebooks are good for illustrating what you did and what you got. They are not good for illustrating how you did it.

It truly is fascinating that you continue to insist that I don't know what a docstring is. Notebooks are good for certain things and not good for others. This conversation is going nowhere.

> - I get an analysis that I like, but there isn't a good way to share it with others, so I end up just taking screenshots.

Jupyter supports printing out as a PDF, an all-in-one-webpage and all sorts of other formats. I've used this and the support for Markdown to create a number of documents that I've presented up the leadership chain, or to other engineers across the org "Here's the data, here's the narrative, and here's the code that shows you how I got to it so you can reproduce". Can even share the Jupyter notebook itself.

Do you have a way to store data directly into the notebook? I do a bunch of device testing and I have master notebooks setup to analyze and condense raw data. I use papermill to evaluate the master notebook to generate a report. But I want to also store/attach intermediates (say a named numpy array) into the notebook for further analysis.

> I want to also store/attach intermediates (say a named numpy array) into the notebook for further analysis.

Is the issue that you do not want to save the data and report into a folder and distribute that? That is, you want an entirely self-contained notebook? Or is there something else going on here?

I'm sure that's possible but it seems kind of wrong to put your binary "data" in with your analysis and presentation-making code.

Somewhat. I want it to be difficult to separate the data from the report. Basically I want the report itself to be ingestible as input to other steps, so it's more of a "documented data" with the analysis results available. The inputs are documented but the final results can be restored without reevaluating the entire notebook. I don't want the entire workspace saved, just the final results. I hoped there was some magic to inject and restore a python object from a cell using some form of introspection.

The testing I do is annual equipment performance evaluation. I'd like to be able to process each test and then feed the results into longitudinal monitoring. One thing I am considering is adding a library or extension to papermill that automatically creates a workspace hdf5 or dill or whatever that I can store individual variables into. After studying the ipynb JSON it just seems odd that you can't just store blobs as attachments. But what I understand is it has to do with the kernels and notebooks running as separate processes and passing things around as notifications. So basically the kernels don't have any access to the cells or any sorts of introspection.

With papermill you have parameters, there's just not any "return values" in the processed notebooks. If it existed you could treat "reports" more easily as cached function evaluations.

It looks like there’s a %store command in Jupyter. I haven’t tried it out, but is this what you are looking for?

https://stackoverflow.com/questions/34342155/how-to-pickle-o...

(I just got it by googling “pickle an object in jupyter,” so sorry if this is something obvious that you’ve already seen and doesn’t quite solve your problem).

Yeah it got my hopes up when I found it. But when I was testing it, I didn't find the data actually made it into the .ipynb. It turns out that's actually a global storage in your home directory and doesn't go into the notebook at all. So different notebooks each overwrite the value if they use the same variable name.

Oh wow, that’s a really awful design. Sorry.

The problem with that is that Jupyter requires a back-end to do any of the actual processing that runs the code in the cells.

Only way I can think to make the html file truly portable while fully functional would be to embed a python interpreter and all required libraries as wasm.

Could be possible with pyodide? I haven't used it.

I just want selected/declared "final results" to be restorable in a new kernel. Papermill let's you easily parameterize notebooks and execute them. So the notebook feels like a well documented functions that behave like executibles with really fancy debug print logs that can be reviewed for correctness and reproducibility.

Papermill does a great job of recording the input parameters and execution history. But there isn't anything equivalent to the "return foo, bar" part of a function which makes it difficult to build up modules. You don't want to have to digitize a plot to carry on to the next step is what I'm saying.

I'm always hesitant to self-promote in an hn comment (actually I have never done it before!), but your problems w/ Jupyter are just too closely mapped to what Hex (https://hex.tech/) solves to not plug it here!

- I get an analysis that I like, but there isn't a good way to share it with others, so I end up just taking screenshots.

You can publish any Hex notebook with literally just a few clicks, and anyone you share it with can access it, or edit it, or fork it, without installing anything— or you can even make it public. You can easily turn a notebook into an "app" or interactive report if you want, hiding/showing certain cells or choosing cells to show only code/only output. You can just share the raw notebook though too.

- There isn't a good way to take the same analysis and plug new data into it, other than to copy-paste the entire notebook.

Super easy to duplicate a Hex project and hit a different table or data source, or you can use input parameters (like ipywidgets) to make one notebook parameterized and work on a bunch of different data sources.

- The process to "promote" fragments of a notebook into being reusable functions seemed very high-friction: basically you're rewriting it as a normal Python package and then adding that to Jupyter's environment.

You can promote any part of a project to a "Component" (docs: https://learn.hex.tech/docs/develop-logic/components) that you can import into other projects. They can be data sources, function definitions, anything. If you make upstream changes to the component, you can sync them down into projects that import it.

- There aren't good boundaries between Jupyter's own Python environment, and that of your notebooks— if you have a dependency which conflicts with one of Jupyter's dependencies, then good luck.

Hex has a ton of default packages in its already installed standard library, and all the dependencies are ironed out— if you have packages you want to use that aren't there, you can pip install them, pull them in from a private github repo, or ask us to add them to the base image. You can also run Hex projects using a custom-provided docker image if you have super custom needs.

You should *definitely* check it out if you have these pain points. Here's an example of a pretty complicated public Hex project: https://app.hex.tech/hex-public/app/9b882bc1-ead3-4f0b-87d1-...

And here's a simpler one I just made the other day on a cool Silk Road dataset https://app.hex.tech/hex-public/app/cdc1b8fe-144b-4a74-a5ef-.... There's a bunch more examples at https://hex.tech/use-cases. Happy to answer any questions!

I was going to post hex as an ideal option, I definitely do not miss dealing with jupyter and all the Python env related headaches, hex solves all of these and then the rest of the more UX related issues with notebooks. Definitely recommend as the best compromise for notebook based Python data analytics tasks

For a while I viewed Jupyter as a toy that is neither here nor there (sitting between the chairs of development and explanation, briefing or visualization and not doing either job great). But about 2 years ago when I changed jobs into a "Jupyter heavy" environment I was forced to learn it and have grown to really like it.

I primarily use Jupyter for prototyping: trying ideas, plotting results and sharing notebooks for others to improve on (or punch holes in). Once a piece clearly shows promise I move it into a module. While this usually means a significant rewrite I actually see it as a benefit: I can be messy in original prototyping plus a rewrite after experimentation often leads to a better code with a small time investment.

Beyond the minimal self-discipline of actually moving code into modules I still have two three-character friction points: vim and git. Pointers on addressing those appreciated!

I would love, love, love a tight vim and Jupyter integration to be able to switch, easily and frequently, between editing a set of cells in vim and in Jupyter with solid sync between them. I am perfectly OK with vim ignoring the output. And I would love to have a git mode that only checks in the changes that cause actual differences in the python code; not timestamps or the output.

jupytext will solve the git problems for you. Pair with a percent script, ignore the ipynb files.

jupytext is great. It even allows you to use only .py files directly as notebooks, but I recommend "pair with ipynb" and version controlling the .py file. The ipynb acts like a cache of the cell outputs between invocations of jupyterlab, which is handy too.

I always wanted a Jupyter like environment, but the one that would natively support and output .py files

jupyter notebook as IDE, but with .py files instead of .ipynb

This is exactly what jupytext gives you. The "pairing" is optional (and sometimed confusing). You can just work with .py files as notebooks and never ever see a .ipynb on your disk again.

The notebook .py files are just regular python files with comments that can be edited at hand wih any text editor. Thus you can easily collaborate with your local graybeards that will dislike editing text on their web browsers.

What I miss is a notebook launcher button that creates a .py notebook. Don't understand why we don't have that (i could not myself find a way to configure and add it.)

I'm not sure why you think it's viable to collaborate with others who don't use jupyter? I mean on any software project, if we don't agree how to compile or run a project, then we usually can't collaborate IMO. Changes become nonsensical (breaking the one or other mode that is not tested by the author of the change.)

I think collaborating with .py files is easier over git.

I could not find a way how to check-in .ipynb file and let reviewer review code only, and ignore the metadata part

if jupytext will allow me to work in jupyter, check in file to git and create MR and let other people easily see, review and comment my code that would be great

Great, then I'd recommend using jupyterlab 3.6 with jupytext right now

This is a plugin that makes browser text area elements into nvim buffers.

https://github.com/glacambre/firenvim

Haven't tested it in combination with Jupyter but I imagine it should work

Are there some usecases where conventional IDE isn't up to the job as the notebooks? I can imagine this to be so for data analysis where pre-loading heavy datasets saves time. Anything else?

For me its the matter of actually liking The notebook environment rather than overcoming the shortconings of IDEs.

The notebook environment is excellent for data analysis and exploration. It matches my workflow of manhandling data as I reach for understanding.

Once I figure things out of course, either I'm done with the notebook and can copy a few plots out for inclusion in a powerpoint, or I'm done with the notebook and extract a pile of functions into a utility script or package.

Either way, it has served its purpose for rapid prototyping and one-off analysis.

You can comfortably edit the whole thing as Markdown in VSCode (in vim mode, if you'd prefer) - and still have executable cells and nice pdf/html rendering. https://quarto.org/docs/tools/vscode.html

EDIT: actually, you can do it directly in neovim - https://quarto.org/docs/tools/neovim.html

This seems like a nice approach. Mine is similar. I refactor often, by moving stuff from a lower level of discipline to a higher one:

Enclose scripts in functions within the notebook, which minimizes the clutter of hidden state. I also have a habit of not walking away from a notebook without doing a "restart kernel and run all cells" to make sure the notebook works. I'm not dealing with giant data sets, so this doesn't cost me much.

Frequently used functions go into .py files, using auto-reload to keep things synchronized while I'm working on them.

Mature .py files that I might want to re-use in different projects get turned into pip-installable packages. The notebooks become informal tests of the packages.

I've never used venv, and never encountered dependency version problems. Some of the dependency horror stories may be obsolete due to the maturation of the big packages such as numpy and matplotlib.

There are lots of different versions of this. One of these approaches may work for you https://alpha2phi.medium.com/jupyter-notebook-vim-neovim-c2d...

This is my preferred way to work as well. Some techniques to address what you are missing:

1. Only checking in semantic differences (not output, timestamps, etc.):

Use the `jupytext`-extension [0] to seamlessly pair you notebook with an lightweight markup version of the input only (which can be used to generate full notebook).

2. Being able to switch between text editor and notebook interface

As other have mentioned, there are integrations for multiple editors.

Another approach is to move the central code out to a python module which you edit in a text editor, and then use the `%autoreload` magic [1] to reimport that module whenever you execute a cell in the notebook.

[0]: https://jupytext.readthedocs.io/en/latest/install.html [1]: https://ipython.readthedocs.io/en/stable/config/extensions/a...

One thing that I keep running into myself is I want to include data in a notebook as a sort of report or record of an analysis. But you can only include text forms or plots. If I wanted to store a numpy array within the notebook there doesn't seem to be a way. I understand it could be difficult to figure out how to make the all of that work. But I don't know. Maybe pickling or something should work?

Ploomber does this kind of proper jupyter notebook pipeline. I've used it, but not stuck with it (yet)

I'll have to check it out. I have looked at it previously but it's somewhat overwhelming to figure out what it actually does. My sense previously was that is it was more like a dependency graph thing similar to a makefile but I'll give it a closer look.

We looked into many of these issues with Deepnote (YC S19) [https://deepnote.com/]. What we found is that these are not necessarily problems of the underlying medium (a notebook), but more of the specific implementation (Jupyter). We've seen a lot of progress in the Jupyter ecosystem, but unfortunately almost none in the areas you mentioned.

>- There isn't a good way to take the same analysis and plug new data into it, other than to copy-paste the entire notebook.

Cant you just set the parameters in the first cell?

>- I get an analysis that I like, but there isn't a good way to share it with others, so I end up just taking screenshots.

Export to md, pdf or html?

I manage a Jupyter instance where we do ~75% of all of our work, and before that used Jupyter daily as a quant. In response to your comments + sibling's:

Sharing: there hasn't been a good solution, so you have to hack something together. For us (small team) symlinking a "shared_notebooks" network path works fine (but looking forward to the RTC mentioned in the post). It's very important to restart your notebook before sharing.

New data: I've never really run into this in a form that "restart + run all cells" didn't really work. Are you trying to keep older versions?

Reusability: I highly recommend installing a personal Python package in development mode to your kernel (i.e. pip install -e .). Then just move functions from the notebook to the package and reload.

Boundaries: As others have said, I would just make a "user" kernel right away and never touch the Jupyter environment (as TLJH does by default).

I am kind of hopeful that Armin Ronacher's rye fixes Python environment hell and we end up with better solutions for many of these things, but there are definitely some issues with version management in Jupyter.

Have you considered using Pluto notebooks instead? They are designed to be reproducible (including version references to transitive dependencies). They are also reactive, so it is trivial to change the dataset.

Export to HTML is usually a decent report. If you want to refine it a bit, you can export and show only outputs, or (with some code) only certain tagged outputs.

To plug in new data, I experiment with papermill.

There are solutions that can put jupyter and the kernel in different environments. It's even not too hard to setup, but it astounds me it's not a common way to set it up. It's natural to have kernel + the analysis's deps in its own environment.

You may be interested in papermill to address the parametrized analysis problem [1]. I think (but I'm not positive) this is what the data team at a previous job used to automate running notebooks for all sorts nightly reports.

[1] https://papermill.readthedocs.io/en/latest/#

If you're looking for a way for others to reproduce your environments, you might find this useful: https://jupyenv.io/

To promote functions to a module use nbdev; execnb or papermill to parameterise your notebook; and nbconda to have environments for your notebooks.

I've thought about this a lot in the context of our org. What you're describing is taking modeling -> production (minus the screen shot one).

Typically a data engineer will be the one who helps bridge that gap, but that has a problem of does data engineer's output == scientist output, which can be time consuming to handle.

To shrink the gap from dev->prod - we have 2 notebooks, one for model development and one model deployment in production. We use papermill[0] to execute directly notebooks in production.

Shared functions between the dev/prod that are built by the scientist are put into a separate notebook and then imported via `run`. If I'm honest, our scientist don't do this and simply copy/paste the functions if we don't yell at them to fix it.

This basically allows us to stay within the jupyter environment entirely so that dependancies are isolated.

So, it's far from perfect, but it's allowed us to shrink the dev->prod life cycle time. Love to hear what others have done towards the same end.

If you use conda there are extensions that can help with this by automatically registering any available conda environments that include ipykernel in your Jupyter Lab environment.

nb_conda_kernels is pretty reliable but not actively maintained. Gator from the mamba folks is new and still a bit rough around the edges but looks like it will be pretty slick eventually.

https://github.com/Anaconda-Platform/nb_conda_kernels

https://github.com/mamba-org/gator

> There aren't good boundaries between Jupyter's own Python environment, and that of your notebooks— if you have a dependency which conflicts with one of Jupyter's dependencies, then good luck.

I believe that you can use https://github.com/tweag/jupyenv for this.

Why not write the experimental code in the notebook, then write a function containing that code in the notebook and try it out, and only then move your function to a py file, if at all?

> I get an analysis that I like, but there isn't a good way to share it with others, so I end up just taking screenshots.

What would be your preferable way of sharing your analysis with others?

- You can turn jupyter notebooks into pdfs directly in the jupyter UI.

- You can upload them to Gitlab/Github and share the link to the rendered result.

- You can upload them to Colab/Binder/Kaggle and let people play with the code themselves.

- You can turn jupyter notebooks into beautiful websites/documents with: https://quarto.org/docs/tools/jupyter-lab.html

- You can add jupyter notebooks to you docs using nbsphinx: https://nbsphinx.readthedocs.io/en/0.9.1/

- You can turn jupyter notebooks into interactive web apps with voila: https://github.com/voila-dashboards/voila

- You can turn jupyter notebooks into presentations with rise: https://github.com/voila-dashboards/voila

I'm guessing the las one should link here https://rise.readthedocs.io/en/stable/ and not to voila two times?

You're right, thanks!

To add to this, shamelessly self-promoting, Notebooker (https://github.com/man-group/notebooker) is a neat way of scheduling your Jupyter notebooks as parametrisable reports whose results are presented in a little web GUI (either as static HTML, PDF, or as reveal.js slideshow renders)

While I have no need for its online functionality and the SAAS part of plotly, I really do like plotly python + cufflinks [1]. It lets you make interactive plots in html/js format. Which means you can save the notebook as html, and while people won't be able to rerun the code, they can still zoom in on graphs, hover to see annotations etc, which is a really nice way to share the outcome of your work in a more accessible way.

[1] https://github.com/santosjorge/cufflinks

If you're interested in an easier way to create reports using Python and Plotly/Pandas, you should check out our open-source library, Datapane: https://github.com/datapane/datapane - you can create a standalone, redistributable HTML file in a few lines of Python.

Cufflinks seems to be stale, maybe it is not needed anymore to bind plotly and pandas? I don't think these options existed in 2021 when cufflinks was last updated:

- Plotly can be used directly as pandas backend: https://plotly.com/python/pandas-backend/

- The plotly.express module makes it easy to create interactive plots in html/js format from pandas dataframes: https://plotly.com/python/plotly-express/#gallery

I think the ideal would be a link directly to the notebook which allows user-local editing/fiddling, with an option to fork the master copy and save it into their own workspace— basically the Github model.

But it sounds from both your comment and the many sibling replies that there are a number of tools now directly addressing this space, so I should definitely re-evaluate what is available.

Do you have an example of how this works with another tool/language?

I don't know if I understood it correctly but maybe you could:

- Upload your notebook to Github, then create a url with Binder (part of the jupyter ecosystem) directly to an editing/fiddling playground: https://mybinder.org/

- If by user-local you mean on their own machine, they can clone your repo and run their own jupyterlab to fiddle

- If everything should stay on your own computer/server, you could share a link to your own jupyterlab and collaborate with others in real-time: https://jupyterlab-realtime-collaboration.readthedocs.io/en/... (doing this securely might be a bit of a hassle)

> - The process to "promote" fragments of a notebook into being reusable functions seemed very high-friction: basically you're rewriting it as a normal Python package and then adding that to Jupyter's environment.

> - There aren't good boundaries between Jupyter's own Python environment, and that of your notebooks— if you have a dependency which conflicts with one of Jupyter's dependencies, then good luck.

The best Jupyter UX for me now is VSCode. Just put an .ipynb file in your workspace and you get the notebook interface inside VSCode. Put `%load_ext autoreload` and `%autoreload 2` in the first cell, and use the same python environment you're using in your workspace for the Jupyter kernel. Then you can import libraries from your project, use them, and it's very easy to promote code from the notebook into a library. You can just cut a function from the notebook, paste it into a library, add an import, and rerun the subsequent cells to verify it still works as expected.