News for hackers. Stuff that matters.
Re:JupyterLab 3.0 by bobbylarrybobby on Tuesday January 05, 2021 @05:16PM (#25651938)
I don't understand why Jupyter notebooks are still in use as a writable format when there are editors like VSCode that can treat ordinary python files as notebooks. The Python extension can submit code blocks to the kernel and import and export Jupyter notebooks, but VSCode is also a real editor on top of that (debugging, git, vim mode, hover info, etc.). For distribution, as a read-only format (like PDF), notebooks are great. But why do people continue to do their work in one?
Re:JupyterLab 3.0 by ajford on Tuesday January 05, 2021 @05:33PM (#25652150)
I don't think anyone who writes production code is using Jupyter as a development env. It's more like a self-documenting, persistent repl, or perhaps an interactive doc.

I've used it as a way to document data exploration for later use and as an interactive teaching tool. For that, being able to plot inline and embed interactive widgets is great. Also, mixing in rendered markdown content.

If you're using Jupyter to write python software and the software is the end result and not the data analysis/process, I think you might be doing it wrong.

Not that I'm discounting data scientists (or the like) who use it as their primary environment. Their end result is not a packaged/deployed software and the code itself is just ancillary. For those users, the code and environment is a tool, and the extra weight of what a full IDE (like VSCode or PyCharm) and the features they bring to the table are unnecessary and likely counter-productive.

Re:JupyterLab 3.0 by teruakohatu on Tuesday January 05, 2021 @06:55PM (#25653015)
> Not that I'm discounting data scientists (or the like) who use it as their primary environment.

Unfortunately it often ends up needing to be executed as a batch job or integrated into a pipeline, which then starts the fun job of extracting the code from the notebook piece by piece.

Jupyter having a plain text file format instead of json would make life so much easier, and vastly improve version control.

Re:JupyterLab 3.0 by boublepop on Tuesday January 05, 2021 @07:32PM (#25653329)
For batch jobs you have papermill, for plaintext format you have the jupytext extension that pairs the Jain notebook with a script with notebook markup in comments. I honestly wish that they had used that format as just being the notebook format, but it’s a nice enough and easy fix.
Re:JupyterLab 3.0 by nojito on Tuesday January 05, 2021 @07:09PM (#25653131)
Re:JupyterLab 3.0 by Jugurtha on Wednesday January 06, 2021 @11:03AM (#25658875)
Not Netflix, but we do use notebooks on our internal platform. Long-running notebooks for model training, but you can view their state as they run even after you close the tab or get disconnected from the internet.

We also publish AppBooks, which are automatically parametrized notebooks. However, we automatically detect and track experiments, and then we deploy models in the form of an endpoint to be invoked by consumers, with a convenience page for humans to invoke the code by sending JSON data, or uploading a CSV file.

- https://iko.ai/docs/notebook/

- https://iko.ai/docs/appbook/

Re:JupyterLab 3.0 by kaba0 on Tuesday January 05, 2021 @05:23PM (#25652019)
Maybe I misunderstand something, but I use jupyter for sage for example and the point of it is that graphical results can be shown there immediately. Some plots can be even made interactive.
Re:JupyterLab 3.0 by bobbylarrybobby on Tuesday January 05, 2021 @05:34PM (#25652162)
When running a Python kernel, VSCode splits the editor into two panes, the input .py file pane and the output ipython pane. The output pane supports interactive graphs just like Jupyter does.
Re:JupyterLab 3.0 by abdullahkhalids on Tuesday January 05, 2021 @06:00PM (#25652429)
1. That's still less intuitive for someone who wants to understand the relationship between the code and output.

2. Following from 1, notebooks are meant to be shared with output in place, so others can understand without executing code.

Re:JupyterLab 3.0 by Jugurtha on Wednesday January 06, 2021 @11:15AM (#25659010)
Exactly. When you want to show results to a client, for example.

>2. Following from 1, notebooks are meant to be shared with output in place, so others can understand without executing code.

We wanted to let others execute the code without being overwhelmed by the code, but we also wanted our colleagues not to write an application or parametrize the notebook by adding metadata or tagging cells, so we added a feature named AppBooks[0]. We automatically detect parameters you want to expose, build a form on top of your notebook, and then serve a clean page.

The user gets a page with a form with fields corresponding to your parameters, they just change the values and hit Run. The notebook runs with the new parameters and the client/stakeholder gets the result.

This solves the problem of: "I have a notebook, how do I not only show my work, but allow domain experts to tinker and run this with specific values". Especially useful when the parameters are pretty domain specific (like instrumentation for nuclear power plants where the client wants to try the model on edge values).

These runs are tracked in case you want to share a training AppBook where it produces a better model, for example.

- https://iko.ai/docs/appbook

Re:JupyterLab 3.0 by alex-treebeard on Tuesday January 05, 2021 @05:31PM (#25652130)
> why do people continue to do their work in one? Most nb users don't use vscode. I'd say main reason is you can't open a notebook without starting a kernel which takes >1 second. It also is harder to setup.

In terms of a flat file formats, there is actually a mark down extension which I would say is better than vscode's format because it is more standardised:

https://myst-nb.readthedocs.io/en/latest/

Re:JupyterLab 3.0 by cat199 on Tuesday January 05, 2021 @06:14PM (#25652574)
just discovered 'jupytext', which is also pretty good for this: https://jupytext.readthedocs.io/en/latest/
Re:JupyterLab 3.0 by cat199 on Tuesday January 05, 2021 @06:13PM (#25652559)
> when there are editors like VSCode that can treat ordinary python files as notebooks

was doing this in emacs before there was a jupyter notebook..

as the name implies, notebooks aren't for development, they are for interactive, shareable presentation, and presentation that also works over HTTP without needing local tooling

Re:JupyterLab 3.0 by klelatti on Tuesday January 05, 2021 @06:06PM (#25652490)
I think you're suggesting that if you want to retain both code and results you need to manage two different files - raw python and exported notebook. I think many notebook users would see that as inconvenient (and many are not familiar with VSCode).
Re:JupyterLab 3.0 by jVinc on Tuesday January 05, 2021 @06:37PM (#25652821)
Jupyterlab has an extention that does this automatically on save, jupytext, also keeps both files in sync over time.
Re:JupyterLab 3.0 by SatvikBeri on Tuesday January 05, 2021 @08:43PM (#25653826)
I use PyCharm + Jupyter Notebooks with autoreload. I'll build up functions in the notebook by running small bits of code, then move them to a Python file. It's like a supercharged version of a REPL.

VSCode's version is nice for sure, but not nearly as nice as being able to go back and read/modify old cells.