Do we need a third Apache project for columnar data representation?

Re:Do we need a third Apache project for columnar data repre... by Notre1 on Tuesday October 31, 2017 @02:18PM (#15595300)

For data frame implementations, you can also look at Spark's Dataset/DataFrame and Graphlab Create's SFrame.

The company/team behind Graphlab Create was bought by Apple and the open sourced components haven't been updated since then. Because of that, I wouldn't use it in production, but if you are just looking for functioning implementations to compare, that gives you one more.

[ Parent | on HN ]

Re:Do we need a third Apache project for columnar data repre... by chubot on Tuesday October 31, 2017 @02:27PM (#15595381)

Thanks, I never heard of Graphlab Create. It is a substantial piece of code! It says it's "out of core", which means it's probably more similar to Parquet/ORC than Arrow. But still interesting.

https://github.com/turi-code/SFrame/tree/master/oss_src/sfra...

For comparison, dplyr and arrow:

https://github.com/tidyverse/dplyr/tree/master/src

https://github.com/apache/arrow/tree/master/cpp/src/arrow

C++ does seem to be useful for stuff like this.

[ Parent | on HN ]

Re:Do we need a third Apache project for columnar data repre... by suresk on Tuesday October 31, 2017 @03:03PM (#15595685)

I thought GraphLab Create had some nice functionality, it is too bad that the Apple acquisition appears to have effectively killed the publicly available product. It doesn't look like you can even pay for the commercial part anymore.

[ Parent | on HN ]