Command-line tools can be faster than a Hadoop cluster (2014)

Re:Command-line tools can be faster than a Hadoop cluster (2... by jbergens on Thursday January 30, 2020 @02:17AM (#22189541)

Some places still seem to go for large Kafka clusters just to calc some stats and forward some messages. I am sure some of their solutions are MapReduce below.

[ Parent | on HN ]

Re:Command-line tools can be faster than a Hadoop cluster (2... by threeseed on Thursday January 30, 2020 @02:30AM (#22189602)

Very curious to understand more about these mythical developers who are recommending a technology/paradigm that stopped being used a decade ago.

You've seen people writing pages and pages of Java code to do ETL ?

[ Parent | on HN ]

Re:Command-line tools can be faster than a Hadoop cluster (2... by doteka on Thursday January 30, 2020 @03:23AM (#22189799)

I worked for a large multinational last year that had multiple teams rolling out terrible Hadoop solutions.

The rest of the world does not particularly care about SV fashion trends.

[ Parent | on HN ]

Re:Command-line tools can be faster than a Hadoop cluster (2... by vonmoltke on Thursday January 30, 2020 @08:53AM (#22191260)

I'm very curious to know where you got this strange notion that MapReduce is "a technology/paradigm that stopped being used a decade ago". I barely knew what MapReduce was in 2010, and didn't touch my first Hadoop cluster until late 2012. Every place I have worked since has had a Hadoop cluster or two.

[ Parent | on HN ]

Re:Command-line tools can be faster than a Hadoop cluster (2... by artimaeis on Thursday January 30, 2020 @08:49AM (#22191237)

I'm on a team of a Fortune500 who uses a combination of Java and C# to do ETL. AMA.

Edit: We're also a fairly greenfield team, with a product in beta that's less than 4 years old. So like, the leaders of the team knowingly crafted us in this particular direction.

[ Parent | on HN ]