News for hackers. Stuff that matters.
Re:Billions wasted on Hadoop startups, the same will eventua... by stingraycharles on Friday May 31, 2019 @06:48AM (#20059743)
Insert "almost" into parent's sentence and it becomes correct.

Many people use/used Hadoop for problems that did not warrant the overhead and complexity that comes with Hadoop. I've seen it countless times with my own eyes that people pre-emptively use tools like Hadoop and Spark because of a chance that they will hit a massive scale in the future.

This happens in both startups and enterprises alike: people like to think they have big problems too often.

Re:Billions wasted on Hadoop startups, the same will eventua... by coredog64 on Friday May 31, 2019 @07:15AM (#20059853)
This. A prior employer spent millions on a Cloudera install that had 5 worker nodes and was being used to service a bunch of generic jobs.

Worse still, it didn’t even use HDFS and we eventually got sick of the crappy embedded Zookeeper/Kafka setup.

Re:Billions wasted on Hadoop startups, the same will eventua... by jabl on Friday May 31, 2019 @07:04AM (#20059811)
> This happens in both startups and enterprises alike: people like to think they have big problems too often.

A.k.a. resume-driven development. Having Hadoop on your CV looks sexier than awk.

Re:Billions wasted on Hadoop startups, the same will eventua... by arethuza on Friday May 31, 2019 @08:20AM (#20060185)
I was at a conference once where someone was presenting and describing something where they used Hadoop to do pretty simple processing on ~30MB of data and the audience appeared to be lapping it up.

I just sat there thinking I could probably run what they did on my phone.

And just to be clear - it wasn't a PoC or a demo.

Re:Billions wasted on Hadoop startups, the same will eventua... by geezerjay on Friday May 31, 2019 @10:49AM (#20061388)
> Insert "almost" into parent's sentence and it becomes correct.

No, it still remains astonishingly wrong. Even container orchestration platforms are being adapted to provide the same service that Hadoop has been providing for years, and no one in their right mind would claim that running processing jobs on the cloud is a problem that almost no one has.