What’s wrong with computational notebooks?

Re:What’s wrong with computational notebooks? by gdy on Tuesday January 28, 2020 @02:21AM (#22167163)

If you have a few long data loading and preprocessing steps it's a pain to wait for them to run again, people try to avoid it.

When something odd begins to happen, they don't immediately consider the possibility that it's not their bug and waste time trying to 'debug' the problem instead of just rerunning the notebook.

[ Parent | on HN ]

Re:What’s wrong with computational notebooks? by playing_colours on Tuesday January 28, 2020 @05:49AM (#22168156)

Would it be a solution to store intermediate computations to an in-memory or disk database like Redis, SQLite? It is a matter of few minutes to run a docker instance and write simple read / write + serialize Python util functions?

[ Parent | on HN ]

Re:What’s wrong with computational notebooks? by gdy on Tuesday January 28, 2020 @08:35AM (#22169188)

Surely, it would be a solution, but I don't think for an average data scientist it's a matter of few minutes.

[ Parent | on HN ]

Re:What’s wrong with computational notebooks? by etangent on Tuesday January 28, 2020 @12:56PM (#22172016)

You don't reload every time you write a line of code. Nobody's insane like that. You reload every two hours or so. This is good enough for most except most extreme data sets.

[ Parent | on HN ]