News for hackers. Stuff that matters.
Re:Google Dataflow: A Unified Model for Batch and Streaming ... by jkff on Sunday October 25, 2015 @04:03PM (#10448166)
Hi! Dataflow team member here.

On maturity: Dataflow is built by the people who over the past 12 years created MapReduce, FlumeJava and FlumeC++, Pregel and Millwheel, based on the sum of experience obtained from all of these. It shares most of the back-end stack (work scheduling, pipeline optimization, fault tolerance etc.) with FlumeJava and FlumeC++ for batch jobs and Millwheel for streaming jobs, all of which are extensively used inside Google for data processing, and it shares a lot of the Java SDK with FlumeJava.

On usage: the blog post that announced General Availability of Dataflow lists a few public customer testimonials: http://googlecloudplatform.blogspot.com/2015/08/Announcing-G... . Quite a few blog posts by other users can also be found via https://www.reddit.com/r/dataflow .

Happy to answer additional questions. If I forget to check on this thread, feel free to ask on [email protected] or on StackOverflow with tag google-cloud-dataflow - we constantly monitor these and usually answer everybody.

There are no comments on this comment yet.