JupyterLab: the next generation of the Jupyter Notebook

Re:JupyterLab: the next generation of the Jupyter Notebook by yomritoyj on Friday July 15, 2016 @07:43AM (#12100279)

I love how notebooks allow the mixing of code and output and support incremental development by letting you choose which cells to execute. But I find the semantics horrible. Each time you execute a cell you do so in an environment that depends on your entire history and cannot be figured out by simply reading the notebook. I wish for a environment which would have the same semantics as a script but which would snapshot the environment at the entry to each cell so that when a cell is modified execution does not have to resume at the beginning. Even better if downstream data dependencies are tracked so that after modifying and reexecuting a cell we know which downstream results have become stale. Does such an environment exist?

[ Parent | on HN ]

Re:JupyterLab: the next generation of the Jupyter Notebook by mjlm on Friday July 15, 2016 @08:44AM (#12100542)

I think this persistent state is one of the main advantages of the notebook environment, or the Matlab workspace, which I guess it was inspired by. It allows you to quickly try alternative values for certain variables without having to re-calculate everything. Saving snapshots would not be feasible if the project contains large amounts of data. If you want to reset everything, just "run all" from the beginning, or use a conventional IDE with a debugger.

[ Parent | on HN ]

Re:JupyterLab: the next generation of the Jupyter Notebook by rhodin on Friday July 15, 2016 @10:40AM (#12101283)

No, not Matlab but Mathematica: "We were inspired originally by the excellent implementation in Mathematica" [1].

[1] http://ipython.org/ipython-doc/dev/whatsnew/version0.12.html...

[ Parent | on HN ]

Re:JupyterLab: the next generation of the Jupyter Notebook by brians on Friday July 15, 2016 @12:18PM (#12102060)

And that came from Emacs and old Lisp environments---and perhaps something yet earlier?

As late as 2000, this was the single biggest advantage and single biggest impediment to new programmers in MIT's 6.001 lab: a bunch of nonvisible state, mutated by every C-x C-e. The student has tweaked two dozen points trying to fix a small program, and re-evaluated definitions after many of them, but maybe not all. The most straightforward help from a teacher is to get the buffer into a form such that M-x eval-region paves over all that, sets a known environment of top level definitions, and---more than half the time---the student's code now works.

I have similar concerns about much of Victor's work, for the same reason. Managing a mental model of complex state is a n important skill for programming, but it's best learned incrementally over long experience with more complex programs. These very interactive environments front load the need for that skill without giving any obvious structure for helping the student learn.

Contrast Excel and HyperCard, which have no invisible state: you can click and see everything.

[ Parent | on HN ]

Re:JupyterLab: the next generation of the Jupyter Notebook by yomritoyj on Friday July 15, 2016 @09:12AM (#12100670)

But you cannot recalculate if your calculation has trashed your inputs.And if it hasn't then the snapshot does not impose a cost. If you are willing to forego the opportunity to replay to save memory, just put the producer and consumer in the same cell.

[ Parent | on HN ]

Re:JupyterLab: the next generation of the Jupyter Notebook by lqdc13 on Friday July 15, 2016 @09:24AM (#12100737)

It is essentially already that if you are careful with the variables.

Currently, you can easily control which things are saved and which aren't. So it is the best of both worlds.

[ Parent | on HN ]

Re:JupyterLab: the next generation of the Jupyter Notebook by 98Windows on Friday July 15, 2016 @08:13AM (#12100404)

That's why I've just resorted to using 'Restart & Run All' as my default way to run a notebook I've finished coding up.

[ Parent | on HN ]

Re:JupyterLab: the next generation of the Jupyter Notebook by denfromufa on Friday July 15, 2016 @12:30PM (#12102142)

What you describe looks like reactive programming and lazy evaluation and Mathematica has support for this using the notion of Dynamic. This is how actually Manipulate is implemented.

[ Parent | on HN ]

Re:JupyterLab: the next generation of the Jupyter Notebook by cdavid on Friday July 15, 2016 @09:11AM (#12100668)

tonicdev seems to do it, though I have never used it: http://blog.tonicdev.com/2015/09/10/time-traveling-in-node.j...

note: it does undo, but not recalculation on demand ala excel.

[ Parent | on HN ]

Re:JupyterLab: the next generation of the Jupyter Notebook by dalailambda on Friday July 15, 2016 @09:49AM (#12100888)

I imagine with a little introspection, a notebook can figure out dependencies between cells and cascade as needed.

[ Parent | on HN ]

Re:JupyterLab: the next generation of the Jupyter Notebook by yomritoyj on Friday July 15, 2016 @10:16AM (#12101075)

I guess the challenge is to do it efficiently in the presence of pointers, implicit or explicit.

[ Parent | on HN ]

Re:JupyterLab: the next generation of the Jupyter Notebook by brians on Friday July 15, 2016 @12:20PM (#12102070)

There's great work on this called differential dataflow. You have to trace at a finer grain than cells, but can then avoid lots of recalculation.

[ Parent | on HN ]

Re:JupyterLab: the next generation of the Jupyter Notebook by thanatropism on Friday July 15, 2016 @12:54PM (#12102309)

The real simple thing would be to paint a border or just a sideline in cells that have already been executed.

Just that would make everything much, much better.

[ Parent | on HN ]

Re:JupyterLab: the next generation of the Jupyter Notebook by jasongrout on Friday July 15, 2016 @02:45PM (#12103007)

Thanks, that makes a lot of sense. I've opened an issue: https://github.com/jupyter/jupyterlab/issues/405.

[ Parent | on HN ]