News for hackers. Stuff that matters.
Re:Readings in Database Systems, 5th Edition (2015) by jrumbut on Sunday June 14, 2020 @01:56AM (#23515448)
I think it's a bit of a shame that the MapReduce concept got the shiny object treatment since I thought it was a nice pragmatic approach to a useful set of problems that are faced all the time and often addressed with ad-hoc programs that make a mess.

People always looked down on those that used Hadoop or somesuch for <1GB of data, but while it wasn't needed from a technology perspective it gave a structure to the project.

Now many places are back in the world of one-off scripts, and I think something of value was lost (even if it was a little ridiculous to fire up a cluster for something Excel or SQLite could handle).

Re:Readings in Database Systems, 5th Edition (2015) by throwaway_pdp09 on Sunday June 14, 2020 @06:19AM (#23516508)
> People always looked down on those that used Hadoop or somesuch for <1GB of data, but while it wasn't needed from a technology perspective it gave a structure to the project.

What 'structure'? Why is it so important that it makes it worthwhile firing up a large, complex framework? I'm beyond baffled.

Re:Readings in Database Systems, 5th Edition (2015) by cbcoutinho on Sunday June 14, 2020 @07:26AM (#23516864)
The same 'structure' that makes it easy to onboard new co-workers because they've seen the same project 'structure' before in the past. In that sense, the bottleneck in an organization is getting people productive as fast as possible, even that means using a cleaver instead of a scalpel.
Re:Readings in Database Systems, 5th Edition (2015) by throwaway_pdp09 on Sunday June 14, 2020 @07:55AM (#23517025)
If all they can use is a massive cleaver (big data tools), and have no experience with scalpels (small, sharp, cheap and fast data tools), IMO your company has a serious, fundamental and systemic problem (no, let's call it failure) towards employee experience, training and knowledge. Edit: and resource management.
Re:Readings in Database Systems, 5th Edition (2015) by _jal on Sunday June 14, 2020 @11:27AM (#23518367)
Seems to be a sort of inverse of the massive spreadsheets that run supply chains on accretions of spaghetti-macros.

But, a tree chipper can serve as a paper shredder, and I imagine a lot of shops in certain markets saw it as a sort of prestige asset around 5-8 years back, when a bunch of companies started hiring data scientists for no apparent rational reason.

(Not bashing data scientists or data companies. Just remembering the fad that went around Bay Area companies a while ago.)

Re:Readings in Database Systems, 5th Edition (2015) by o1lab on Sunday June 14, 2020 @06:23AM (#23516536)
>> (even if it was a little ridiculous to fire up a cluster for something Excel or SQLite could handle)

I know above comment will be lost - but this is such a genuine truth.

Re:Readings in Database Systems, 5th Edition (2015) by ses1984 on Sunday June 14, 2020 @09:13AM (#23517427)
I'm sorry but if your problem can be solved on excel then hire people who are good with excel, not people good with Hadoop.
Re:Readings in Database Systems, 5th Edition (2015) by MaysonL on Sunday June 14, 2020 @11:10AM (#23518215)
Things tend to evolve - a system that worked on excel yesterday may well be a dangerous un maintainable monster next week.