The `export` command can generate a rendered artifact if that's what you're after but there is also another avenue here, have you seen the caching feature? The one that caches to disk and persists?
https://docs.marimo.io/guides/expensive_notebooks/?h=cache#d...
This can automatically store the output of expensive functions, keeping the previous state of the cells in mind. If a re-compute ever needs to happen it will just load it straight from cache.
Another option is to run in lazy mode, documented here:
https://docs.marimo.io/guides/expensive_notebooks/?h=cache#l...
This will prevent the notebook from rerunning cells by accident.
We're thinking about adding features that would make marimo great for running long running batch work but there's not a whole lot I can share about it yet. If you have specific thoughts or concerns though, feel free to join our discord!
If I understand it correctly, `@mo.persistent_cache(name="my_cache")` creates a binary file `my_cache` that I should commit as well if I don't want others to repeat the computation?
This kinda solves the problem, except for having two files per notebook, and that marimo notebooks are no longer viewable with output on github directly.
We plan to add more stores like Redis, S3-bucket, or an external server, since you may not always want to commit this file, but like you said want others to avoid the computation.
Is that really what's wanted? If there's some cell that I need to run twice for some reason that I tried debugging but wasn't able to figure out why, or for debug just run cells 1-5 in order but on specific other (prod) systems, skip cell 4 and run 3 before 2. Now, arguably well written software would handle those things automatically, but we're not talking about battle hardened software that's had an SRE team vigorously refactor it until it's been proven suitable for such purpose by having been on call for it for months off not years. We're talking about notebooks, which have their time and place, but the entire point, I would argue, is to make it easier run notebooks in production without the added overhead of said SRE team. And in that world, the reality is the PhD is gonna have some things they know they should fix, but it's easier to just comment out the unneeded cells on the prod and hit run all, so shouldn't the tools better support that use case (by saving what was actually run and offering to rerun that) over caching based on input string and hoping for the best?
Arguably this is a good thing. You shouldn't distribute things you can't prove have the same results and one way to do that is to require others to run the same computations.
Most learners do not need to fact check the instructor, but do want to see the operations which were run. Those that are curious can run/edit the notebook themselves.
Edit: The JupyterBook ecosystem (https://executablebooks.org/en/latest/gallery/) as an example of what is possible with stored plots/calculations. Most learners are just going to follow along with the material, but being able to optionally play with the data is the super power of the platform with minimal friction.
I have looked at marimo several times, and while it's great for interactive computing, and it has a fantastic team, it's not a replacement for notebooks and I find their use of the term "notebook" confusing. As a scientist, I don't understand what use case they are exploring, but I do know it's not the use case where Jupyter was created and it's not my current use case for Jupyter on teams I work with.
small edits to code yield enormous Git diffs;
code is copy-pasted across notebooks, instead of reused;
magic commands limit the portability of notebook code;
logic that would be useful as a script or library gets thrown away;
logic that should be tested almost never is.
These are problems primarily a software engineer has. Not problems a scientist thinks are important. If you asked a scientist to list problems with Jupyter notebooks (and there are many), it would be very different list, primarily about science.Why not? Can you expand on this because I don't see why this is not a good thing.
Besides if you distribute your code alongside your output, aren't you providing that proof anyway? People can run your code and see they are getting the same result.
marimo lets you automatically snapshot outputs in an auxiliary file while you work. We also have a persistent cache that lets you pick up where you left off — saving not just outputs but also computed data.
Many of our users do very heavy computation.
This is not quite true. Outputs are not stored...in the Python file*. marimo does store outputs in the `/__marimo__` folder with settings enabled.
> writing the boiler plate to let the reader load the results.
Therea are some primitives to do this for you, such as mo.persistent_cache. This can be an annotation or 'with' block. It intelligently knows when either the source code or inputs change.
The plan is to take this one step further than storing just the output. Because marimo knows the dependencies of each cell, in a future version, it will store each output AND let you know which are stale based on code changes. This is being built by Dylan (from the blog) and inspired by Nix.
You can work your way around it by storing the results in a separate file(s), and writing the boiler plate to let the reader load the results. Or they let you export to ipynb - which is still sharing two files.
Presumably the reason for this decision is making git diffs short. But to me the solution is to fix git diff to operate on JSON nicely, rather than changing the entire notebook format.