Re:Efficient and Compact Spreadsheet Formula Graphs
by narush on Wednesday February 15, 2023 @08:29PM
(#34813730)
Having only read the abstract, this looks pretty cool! The authors exploit the fact that neighboring cells often have similar formulas to compress the evaluation graph.
(Off topic but very related) I’ve been wondering recently more spreadsheets don’t support a mode of operation where everyone column just has a single formula. Think about e.g. a pandas dataframe - most of the code you write only operates at the column level. For certain data science applications, I think it might be a more sensible and convenient default than every cell being a different formula.
Of course, you can’t do an LBO model with that limitation… but not every spreadsheet need support every use case…
Re:Efficient and Compact Spreadsheet Formula Graphs
by probablypower on Thursday February 16, 2023 @03:53AM
(#34816510)
This seems odd to me. This is basically just ways of finding database-esque tables in spreadsheets to then leverage for vector/matrix operations.
It is a non-problem for anyone who really cares about performance as they've probably already realised they can just use a database some efficiently programmed transformations.
I guess it would be a great help for those poorly-optimised mega-spreadsheets that one can find in the corporate world. I'd say that the solution shouldn't be a smart algorithm to enable further mis-use of excel/spreadsheeting-tools, but rather to migrate the processes to a database. Not only do you get better data storage and computational speed efficiency, you also enable better data governance and access (or access control).
This paper is like adding an extra lane on a highway and making traffic worse.
Re:Efficient and Compact Spreadsheet Formula Graphs
by BenoitP on Thursday February 16, 2023 @05:13AM
(#34816929)
> the solution shouldn't be a smart algorithm to enable further mis-use of excel/spreadsheeting-tools, but rather to migrate the processes to a database
In an ideal world only. Being able to have intricate computation definition being user-modifiable is a big feature. I have successfully 'appified' such Excel sheets in the past, only to be recognized as the new business rule maintenance guy. I no longer do that anymore: I identify moving parts, and these stay in the Excel world instead of going into the DB or a config file. I do clean the inputs and harden access control though.
The apps are quite often low traffic so this setup might cost at most 100 USD/y in wasted performance per app. Nobody tracks that anyway. The business guys are happy to maintain the thing, I'm happy they're not bothering me. What is not so good is the latency of some queries.
IMHO Excel is a great global minimum in the 'Configuration Complexity Clock' [1]. Or maybe I just don't care enough anymore.
Re:Efficient and Compact Spreadsheet Formula Graphs
by snthpy on Thursday February 16, 2023 @04:10AM
(#34816602)
Does anyone know of a tool/library that extracts the computational graph from an Excel spreadsheet? I saw someone start a project on this a few years ago but I don't know if it ever got anywhere.
(Off topic but very related) I’ve been wondering recently more spreadsheets don’t support a mode of operation where everyone column just has a single formula. Think about e.g. a pandas dataframe - most of the code you write only operates at the column level. For certain data science applications, I think it might be a more sensible and convenient default than every cell being a different formula.
Of course, you can’t do an LBO model with that limitation… but not every spreadsheet need support every use case…
Sounds like tables in Excel?
It's almost-required for Power Query.
It is a non-problem for anyone who really cares about performance as they've probably already realised they can just use a database some efficiently programmed transformations.
I guess it would be a great help for those poorly-optimised mega-spreadsheets that one can find in the corporate world. I'd say that the solution shouldn't be a smart algorithm to enable further mis-use of excel/spreadsheeting-tools, but rather to migrate the processes to a database. Not only do you get better data storage and computational speed efficiency, you also enable better data governance and access (or access control).
This paper is like adding an extra lane on a highway and making traffic worse.
In an ideal world only. Being able to have intricate computation definition being user-modifiable is a big feature. I have successfully 'appified' such Excel sheets in the past, only to be recognized as the new business rule maintenance guy. I no longer do that anymore: I identify moving parts, and these stay in the Excel world instead of going into the DB or a config file. I do clean the inputs and harden access control though.
The apps are quite often low traffic so this setup might cost at most 100 USD/y in wasted performance per app. Nobody tracks that anyway. The business guys are happy to maintain the thing, I'm happy they're not bothering me. What is not so good is the latency of some queries.
IMHO Excel is a great global minimum in the 'Configuration Complexity Clock' [1]. Or maybe I just don't care enough anymore.
[1] http://mikehadlow.blogspot.com/2012/05/configuration-complex...
=> https://github.com/vinci1it2000/formulas
This looks really well developed now!