News for hackers. Stuff that matters.

A comment on Joining a billion rows 20x faster than Apache Spark.

Re:Joining a billion rows 20x faster than Apache Spark by Loic on Thursday February 09, 2017 @02:27PM (#13610019)
What is the algorithm used to join the tables? Is it a hash join on `id` and `k` or using the fact that the ids are sorted and using a kind of galloping approach?
[ Parent | on HN ]
Re:Joining a billion rows 20x faster than Apache Spark by jagsr123 on Thursday February 09, 2017 @03:40PM (#13610731)
Yes, it is a hash join.
[ Parent | on HN ]
Re:Joining a billion rows 20x faster than Apache Spark by Loic on Friday February 10, 2017 @03:08AM (#13613940)
I will need to dig into the implementation of the hash function, it must be a nice read as the speed shows that it is definitely well optimized! Thank you.
[ Parent | on HN ]