In our case there’s very little magic, and a dead simple engine (so far). We (from a DS perspective) have found that expressing transforms in this manner simplifies the code making it well worth the added layer of abstraction. As a platform team we’ve focused on debugging (viz, etc…), but this is part of the reason we open sourced it! The more voices we get the more operational concerns we can iron out to avoid painful debugging experiences.
In a spreadsheet you're basically building a DAG (assuming you're using it right) that automatically and efficiently recalculates the downstream nodes whenever you change any of the input nodes.
I think it's actually a pretty hard experience to recreate in many programming languages. I think in Python the thing that comes closest these days is Streamlit, but it can still be a lot slower to put together something than a really fast Excel jockey.
Suppose you had a HamiltonFrame that knew the graph that generated it and knew which columns were inputs and which were outputs. When you update any of the input values it could automatically recalculate any columns downstream of the modified inputs.
This project has similar motivations but explains itself a bit more I think. https://opensource.janestreet.com/incremental/
EDIT: Just realizing belatedly that since it was the author of Tributary* who mentioned it he can probably explain himself better than I can...
*along with roughly 450 cool Jupyter extensions.
This reminds me of an internal Google tool for doing async programming in Java (ProducerGraph or something). The idea was that you'd just write annotated functions and the framework would handle all the async stuff. Wasted many thousands of engineering hours while giving an even worse experience than manipulating futures directly.