News for hackers. Stuff that matters.
Re:Druid: fast column-oriented distributed data store by fangjin on Thursday March 31, 2016 @07:51PM (#11401496)
Druid's main value add to the data infrastructure space is around power user-facing data applications at scale. The queries it is best at are OLAP/business intelligence style queries. It isn't really designed to be a general processing tool such as Hadoop or Spark. The open source data space is very complex, and there are many different solutions targeted towards many different use cases. Druid is better than other solutions at some of these use cases, and worse than other solutions at others.

I wrote my interpretation of the current open source data landscape here for anyone interested: http://imply.io/post/2015/11/04/big-data-zoo.html

Re:Druid: fast column-oriented distributed data store by deepanchor on Friday April 01, 2016 @12:42AM (#11402689)
As someone who hasn't yet had the opportunity to use many of these systems, this was a great high-level overview of how the various systems fit together. Thanks for writing it!
Re:Druid: fast column-oriented distributed data store by scott_s on Friday April 01, 2016 @10:39AM (#11405067)
Your Wikipedia link to "stream processors" is for the wrong kind of stream processors. For a decades, the digital signal processing and graphics worlds have used the streaming abstraction to design programs and hardware. Their applications are typically expressed in pipelines, and do have to continually process data.

"Big data" stream processing is obviously related as its a dataflow programming model, but it's still very different in practice. The streaming abstraction is generally more free-form, and not realized directly in hardware. I contrast the two kinds of streaming in Section 2 of a paper from a few years ago: http://www.scott-a-s.com/files/pact2012.pdf