Re:Readings in Database Systems, 5th Edition (2015)
by throwaway_pdp09 on Sunday June 14, 2020 @06:19AM
(#23516508)
> People always looked down on those that used Hadoop or somesuch for <1GB of data, but while it wasn't needed from a technology perspective it gave a structure to the project.
What 'structure'? Why is it so important that it makes it worthwhile firing up a large, complex framework? I'm beyond baffled.
Re:Readings in Database Systems, 5th Edition (2015)
by cbcoutinho on Sunday June 14, 2020 @07:26AM
(#23516864)
The same 'structure' that makes it easy to onboard new co-workers because they've seen the same project 'structure' before in the past. In that sense, the bottleneck in an organization is getting people productive as fast as possible, even that means using a cleaver instead of a scalpel.
Re:Readings in Database Systems, 5th Edition (2015)
by throwaway_pdp09 on Sunday June 14, 2020 @07:55AM
(#23517025)
If all they can use is a massive cleaver (big data tools), and have no experience with scalpels (small, sharp, cheap and fast data tools), IMO your company has a serious, fundamental and systemic problem (no, let's call it failure) towards employee experience, training and knowledge. Edit: and resource management.
Re:Readings in Database Systems, 5th Edition (2015)
by _jal on Sunday June 14, 2020 @11:27AM
(#23518367)
Seems to be a sort of inverse of the massive spreadsheets that run supply chains on accretions of spaghetti-macros.
But, a tree chipper can serve as a paper shredder, and I imagine a lot of shops in certain markets saw it as a sort of prestige asset around 5-8 years back, when a bunch of companies started hiring data scientists for no apparent rational reason.
(Not bashing data scientists or data companies. Just remembering the fad that went around Bay Area companies a while ago.)
Re:Readings in Database Systems, 5th Edition (2015)
by o1lab on Sunday June 14, 2020 @06:23AM
(#23516536)
>> (even if it was a little ridiculous to fire up a cluster for something Excel or SQLite could handle)
I know above comment will be lost - but this is such a genuine truth.
People always looked down on those that used Hadoop or somesuch for <1GB of data, but while it wasn't needed from a technology perspective it gave a structure to the project.
Now many places are back in the world of one-off scripts, and I think something of value was lost (even if it was a little ridiculous to fire up a cluster for something Excel or SQLite could handle).