> big data workflow involves loading millions and millions of records
Dude, if it fits on your laptop disk, it's not big data. Here you're talking about “big data” that fits in RAM on my cellphone. Somebody needs to get over himself.
But if Google Sheets can't fit the data they are using, it is by definition "big data", even if it's not Big Data. I think this over-protectiveness of the definition doesn't really serve anyone.
It serves everyone to have relatively stable meanings for words and phrases. "Relatively" because of course there is natural drift over time. But if today I write "big data" and it means "millions of rows in Excel" to Bob but "hundreds or thousands of terrabytes" to Jane, it's effectively useless to use in communication.
Put another way, that Bob thinks he's dealing with "big data" doesn't mean he is in fact.
In four lines you manage to be arrogant, insulting, patronizing, and intellectually sloppy (“by definition”?) I'm disappointed. I'd like to invite you to produce a higher level of discourse.
...you can't be serious. This is an incredibly over the top reaction to a very mild reply. Not to mention that telling me you're disappointed in me and inviting me to "produce a higher level of discourse" is far more arrogant and patronising than anything in my original statement.
But anyway, this is very obviously off-topic. I don't think it serves the wider community to continue this conversation so I'll leave it here.
Well, I'm certainly guilty of the same offense!
There's a company in my building that outfitted their team with $30K+ HP Z series workstations with 256GB of RAM... to run Microsoft Excel 2016.
That's more RAM than my cellphone has, but less disk than my laptop. It's getting close to being “big data.”
Right, if it's only millions of records, why can't Google sheets accommodate it.
DHTML involves trading 99.9% or so of your computational power for greater convenience of programming.
Re:Why Enterprise Software Sucks
by probablybroken on Friday October 11, 2019 @10:35AM
(#21224753)
That's really strange... I always found Excel to perform dreadfully on data sets with even just hundreds of thousands of rows - it was always easier to just load the data into a database, or manipulate it with a script.
That's because you don't know how to use Excel (this isn't a knock on you, no one ever tells people how to use Excel). Excel sheets have a hard row limit of around a million rows, but long before you approach that you will have moved your data into the "Data Model", which is a relational table engine stapled on to Excel, or to some other backend which you will query from Excel (often via the Data Model).
The presenter looked confused, and then the group asking the question clarified that their big data workflow involves loading millions and millions of records into excel and they would have trouble migrating to google cloud until google sheets could match excel in performance.
Somewhat unrelated story, but it shows how well loved some software is, and how some users will take it to use cases you never imagined.