Re:Do we need a third Apache project for columnar data repre...
by tshiran on Tuesday October 31, 2017 @12:45PM
(#15594605)
I would argue that the fact that Arrow has been integrated into so many projects over the last year is proof that a separate project made sense. Dremio, PySpark, Pandas, MapD, NVIDIA GPU Data Frame, Graphistry, Turbodbc, ...
[ Parent
| on HN ]
Re:Do we need a third Apache project for columnar data repre...
by rch on Tuesday October 31, 2017 @01:09PM
(#15594783)
Here's the conclusion of the article:
It makes sense to keep Arrow and Parquet/ORC as separate projects, while also continuing to maintain tight integration.
You might enjoy reading it to see why.
Re:Do we need a third Apache project for columnar data repre...
by ineedasername on Tuesday October 31, 2017 @01:15PM
(#15594819)
Well... use in projects is an indicator, not a proof, of utility. Not all choices are good ones, inferior & redundant products are crop up on a regular basis, and to the dismay of many in this community, sometimes win out. Happily, from the article, Arrow, doesn't appear one of them, and has its own unique and justified niche.