News for hackers. Stuff that matters.
Re:Joining a billion rows 20x faster than Apache Spark by usgroup on Thursday February 09, 2017 @03:28PM (#13610606)
Lol was hoping it was a combination of awk and paste :)

That always makes me chuckle.

Honestly though ... Jenkins + bash + cloud storage and you'll be surprised at how many big data problems you can solve with a fraction of the complexity.

Re:Joining a billion rows 20x faster than Apache Spark by pmlnr on Friday February 10, 2017 @04:54AM (#13614351)
Re:Joining a billion rows 20x faster than Apache Spark by makapuf on Thursday February 09, 2017 @04:46PM (#13611314)
Pardon my ignorance but what would you use Jenkins for ? Scheduling ?
Re:Joining a billion rows 20x faster than Apache Spark by tetha on Thursday February 09, 2017 @05:05PM (#13611454)
Jenkins in such a setting gives you two good things: i) scheduling. and ii) access control. The ability to give random dude X the ability to trigger computation Y, Z, and A, without the ability to change said computations.
Re:Joining a billion rows 20x faster than Apache Spark by usgroup on Thursday February 09, 2017 @04:58PM (#13611401)
Triggering jobs more generally (schedule or push notification) and splitting things into jobs and/or pipelines.