News for hackers. Stuff that matters.
Re:Spark as a Compiler: Joining a Billion Rows per Second on... by SixSigma on Monday May 23, 2016 @03:04PM (#11756033)
How difficult was the bug reporting process?

That is a good metric for a project.

Re:Spark as a Compiler: Joining a Billion Rows per Second on... by threeseed on Monday May 23, 2016 @08:58PM (#11758217)
Bug reporting is simple as per any Apache project i.e. JIRA.

But let's be clear here Spark is not a 'real' open source project. It's a dictatorship run by Databricks (in the nicest possible way). There are 414 outstanding pull requests some of which came from the likes of Intel which added subquery support over a year ago. Never got merged.

The best metric for any open source project is how easily can you contribute and improve the product.

Re:Spark as a Compiler: Joining a Billion Rows per Second on... by kod on Tuesday May 24, 2016 @11:54AM (#11762560)
I totally agree that's the best metric. I totally disagree with your characterization of Spark.

I showed up on the mailing list, said "hey, here's some stuff that was useful for me, let me know if it's useful for anyone else." A committer said, "cool, send a PR". Couple of rounds of code review and it was merged. Later on, bigger features required a lot more discussion, but as long as I was willing to follow up, I got my changes in.

Spark has on the order of 1,000 contributors. I'd call that a "real" open source project.

Re:Spark as a Compiler: Joining a Billion Rows per Second on... by rxin on Tuesday May 24, 2016 @03:40PM (#11764854)
Please keep the contributions coming!