Yahoo Open Sources S4 Real-Time MapReduce Framework

Re:Yahoo Open Sources S4 Real-Time MapReduce Framework by rektide on Wednesday November 03, 2010 @02:36PM (#1865887)

the entire point was that it's not mapreduce, it's a stream processing system. try http://wiki.s4.io/Manual/S4Overview#toc1 for a "what is S4" intro.

The architecture resembles the Actors model, providing semantics of encapsulation and location transparency, thus allowing applications to be massively concurrent while exposing a simple programming interface to application developers.

and

S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable platform that allows programmers to easily develop applications for processing continuous unbounded streams of data.

go stream processing. also see Flume for related recent works.

[ Parent | on HN ]

Re:Yahoo Open Sources S4 Real-Time MapReduce Framework by DanielRibeiro on Wednesday November 03, 2010 @04:22PM (#1866331)

Seems very much like Percolator from google: http://www.infoq.com/news/2010/10/google-percolator

[ Parent | on HN ]

Re:Yahoo Open Sources S4 Real-Time MapReduce Framework by schumihan on Wednesday November 03, 2010 @10:22PM (#1867355)

I don't think so. Percolator does not support stream operators, and it supports transaction semantic. Streaming process engine like s4 will never handle the race conditions brought by multiple writers.

[ Parent | on HN ]

Re:Yahoo Open Sources S4 Real-Time MapReduce Framework by atarashi on Wednesday November 03, 2010 @09:30PM (#1867215)

Indeed, it's not really MapReduce (as in Hadoop or Google's MapReduce), but Yahoo Labs did call it "Real-Time MapReduce."

http://labs.yahoo.com/event/99

[ Parent | on HN ]