News for hackers. Stuff that matters.
Re:Scaling our spreadsheet engine from thousands to billions... by mirntyfirty on Wednesday July 06, 2022 @10:39AM (#32001675)
Can you elaborate some of the use cases for spreadsheets having billions of rows? Databases of phone calls and credit card transactions reach this size, but I would never imagine someone wanting to dump any significant amount of transaction data into a spreadsheet without quite a bit of SQL/R/Py modification ahead of time.
Re:Scaling our spreadsheet engine from thousands to billions... by anyfactor on Wednesday July 06, 2022 @11:48AM (#32002697)
I run a VA business with my brother so we deal with a lot of spreadsheets. We have had a few real estate firms as our customers. They put EVERYTHING in Google Sheets and Excel.

Everything from regulatory document , CRM data and checklists everything. Having billions of row might sound problematic but when a google sheet gets capped or slow they will either create a new sheet on the same file or a new file. A RE firm over its 10 year operations has hundreds and hundreds of spreadsheets. They even have spreadsheets to locate spreadsheets.....

Having everything in one spot is a slightly better thing to do. You can't pry spreadsheets from small businesses no matter what justification you have. From a practical standpoint upping the limits is kind of a good thing.

We usually clean everything up manually and create a proper database that they can query easily. Or they can just tell us what they need through email.

Re:Scaling our spreadsheet engine from thousands to billions... by willdearden on Wednesday July 06, 2022 @11:56AM (#32002809)
Veterans Affairs? Virginia?
Re:Scaling our spreadsheet engine from thousands to billions... by anyfactor on Wednesday July 06, 2022 @01:19PM (#32003979)
Virtual Assistance.
Re:Scaling our spreadsheet engine from thousands to billions... by exp1orer on Wednesday July 06, 2022 @01:08PM (#32003855)
Maybe virtual assistant?
Re:Scaling our spreadsheet engine from thousands to billions... by supersync on Wednesday July 06, 2022 @02:31PM (#32004925)
Fascinating. We’re seeing the same thing. Seeing a few companies opt for Coda as a way to convert some sheets to processes. Here’s what we’ve done with it: https://supsync.com/squared-away/
Re:Scaling our spreadsheet engine from thousands to billions... by Lukas1994 on Wednesday July 06, 2022 @12:17PM (#32003110)
E-commerce companies have 1000s of products, modelled over 100s of weeks, broken down by 10s of locations, ... SaaS companies running cohort models over multiple products, geographies, ...

You pretty quickly end up with very large numbers.

Re:Scaling our spreadsheet engine from thousands to billions... by mirntyfirty on Wednesday July 06, 2022 @07:13PM (#32008054)
What’s the best etl process for transferring the data?