News for hackers. Stuff that matters.

A comment on Apache Flink. Parent comment

Re:Apache Flink by rsynnott on Wednesday January 30, 2019 @09:05AM (#19035085)
In the beginning, there was Hadoop, which was just a MapReduce clone, more or less.

And then there was Hadoop 2, which was kind of meant to do everything.

And that didn't really work out, so there was specialisation.

I don't really see the problem. Many of these do different things, others do similar things different ways. Some are essentially dead ends (I doubt there are many new users of Storm, say). This all seems pretty normal.

[ Parent | on HN ]
Re:Apache Flink by justicezyx on Wednesday January 30, 2019 @10:29AM (#19035801)
No Hadoop does more than MapReduce, much more. Primarily: MapReduce does not have a distributed file-like block storage.
[ Parent | on HN ]
Re:Apache Flink by bunderbunder on Wednesday January 30, 2019 @10:41AM (#19035912)
OK, so, to be more precise:

Hadoop MapReduce does what Google MapReduce does, and Hadoop Distributed File System does what the Google Filesystem does.

And Apache Spark does the same thing as that other Google product whose name I forget.

And Drill vs Dremel, etc etc etc.

I think that the OP is plenty right for the purposes of the point they were making. And I think (from a non-Googler's perspective) that it's a worthwhile point, since the FOSS ecosystem has since evolved along similar lines to Google's internal tools.

[ Parent | on HN ]
Re:Apache Flink by erik_seaberg on Wednesday January 30, 2019 @03:33PM (#19039141)
Apache Spark does what Google FlumeJava does, express a chain of map/reduce steps in a DSL (and inline away some intermediate results).

Apache Flume is unrelated except for the confusing name.

[ Parent | on HN ]