Re:Command-line tools can be faster than a Hadoop cluster (2...
by commandlinefan on Thursday January 30, 2020 @10:12AM
(#22191826)
> in favour of Hive, Spark
Well, Hive and Spark create MapReduce jobs to satisfy your queries: it's still in the background, but you don't have to think about it (as much).
Re:Command-line tools can be faster than a Hadoop cluster (2...
by jbergens on Thursday January 30, 2020 @02:17AM
(#22189541)
Some places still seem to go for large Kafka clusters just to calc some stats and forward some messages. I am sure some of their solutions are MapReduce below.
Re:Command-line tools can be faster than a Hadoop cluster (2...
by threeseed on Thursday January 30, 2020 @02:30AM
(#22189602)
Very curious to understand more about these mythical developers who are recommending a technology/paradigm that stopped being used a decade ago.
You've seen people writing pages and pages of Java code to do ETL ?
Re:Command-line tools can be faster than a Hadoop cluster (2...
by doteka on Thursday January 30, 2020 @03:23AM
(#22189799)
I worked for a large multinational last year that had multiple teams rolling out terrible Hadoop solutions.
The rest of the world does not particularly care about SV fashion trends.
Re:Command-line tools can be faster than a Hadoop cluster (2...
by vonmoltke on Thursday January 30, 2020 @08:53AM
(#22191260)
I'm very curious to know where you got this strange notion that MapReduce is "a technology/paradigm that stopped being used a decade ago". I barely knew what MapReduce was in 2010, and didn't touch my first Hadoop cluster until late 2012. Every place I have worked since has had a Hadoop cluster or two.
Re:Command-line tools can be faster than a Hadoop cluster (2...
by artimaeis on Thursday January 30, 2020 @08:49AM
(#22191237)
I'm on a team of a Fortune500 who uses a combination of Java and C# to do ETL. AMA.
Edit: We're also a fairly greenfield team, with a product in beta that's less than 4 years old. So like, the leaders of the team knowingly crafted us in this particular direction.
Pretty confident that not a single developer anywhere in this world would be first thinking of MapReduce. Just like they wouldn't jump straight to Cobol.