Re:Scaling our spreadsheet engine from thousands to billions...
by theolddirty on Wednesday July 06, 2022 @09:57AM
(#32000972)
Interesting. A few q's as an Excel performance nerd who's fooled around in Pandas:
* Why the design choice to tie the calc engine directly to a DB vs computing dependencies and returning values post-facto?
* How does vectorization scale when using "volatiles" like offset-style formulas that may dynamically change the calc graph?
Every time I tried to recreate stuff in Excel in Pandas/Numpy, 1:1 multiplication wasn't the issue, the weird dependencies that prevented using their vectorized capabilities were.
* Why the design choice to tie the calc engine directly to a DB vs computing dependencies and returning values post-facto? * How does vectorization scale when using "volatiles" like offset-style formulas that may dynamically change the calc graph?
Every time I tried to recreate stuff in Excel in Pandas/Numpy, 1:1 multiplication wasn't the issue, the weird dependencies that prevented using their vectorized capabilities were.