Re:Spark as a Compiler: Joining a Billion Rows per Second on...
by kod on Tuesday May 24, 2016 @11:54AM
(#11762560)
I totally agree that's the best metric. I totally disagree with your characterization of Spark.
I showed up on the mailing list, said "hey, here's some stuff that was useful for me, let me know if it's useful for anyone else." A committer said, "cool, send a PR". Couple of rounds of code review and it was merged. Later on, bigger features required a lot more discussion, but as long as I was willing to follow up, I got my changes in.
Spark has on the order of 1,000 contributors. I'd call that a "real" open source project.
But let's be clear here Spark is not a 'real' open source project. It's a dictatorship run by Databricks (in the nicest possible way). There are 414 outstanding pull requests some of which came from the likes of Intel which added subquery support over a year ago. Never got merged.
The best metric for any open source project is how easily can you contribute and improve the product.