News for hackers. Stuff that matters.
Re:Spark 2.0 Technical Preview by rxin on Wednesday May 11, 2016 @05:30PM (#11679476)
Yup similar to that. Our next blog post in the pipeline is going to reference this paper.
Re:Spark 2.0 Technical Preview by cachemiss on Wednesday May 11, 2016 @05:34PM (#11679509)
(Guessing you meant to respond to me)

Excellent, I'm glad that the "big data" world is starting to look at database literature in terms of how it does execution, as there is much to be learned.

Most of these systems are extremely inefficient (looking at you Hadoop), when they don't really have to be. Efficient code generation should be table stakes for any serious processing framework IMO.

Re:Spark 2.0 Technical Preview by rxin on Thursday May 12, 2016 @01:09AM (#11681441)
Yup was replying to you. Clicked the wrong button :)