News for hackers. Stuff that matters.

A comment on Apache Arrow 3.0. Parent comment

Re:Apache Arrow 3.0 by breck on Wednesday February 03, 2021 @05:40PM (#26020157)
Arrow is definitely one of the top 10 new things I'm most excited about in the data science space, but not sure I'd call it the most important thing. ;)

It is pretty awesome, however, particularly for folks like me that are often hopping between Python/R/Javascript. I've definitely got in on the roadmap for all my data science libraries.

Btw, Arquero from that UW lab looks really neat as well, and is supporting Arrow out of the gate (https://github.com/uwdata/arquero).

[ Parent | on HN ]
Re:Apache Arrow 3.0 by georgewfraser on Wednesday February 03, 2021 @10:17PM (#26022191)
Much of the value of Arrow is in the things that will get built after Arrow is widely supported by data warehouses. Much of the data ecosystem we have today was designed to avoid the cost of moving data between systems. The whole Hadoop ecosystem is written in Java and shoehorned into map-reduce for this reason.

Imagine if, for example, you could use Mathematica or R to analyze data in your Snowflake cluster, with no bottleneck reading data from the warehouse even for giant datasets. This is the future that’s going to be enabled by Arrow.

[ Parent | on HN ]
Re:Apache Arrow 3.0 by RainBoooow on Thursday February 04, 2021 @06:17AM (#26024443)
Is it not what Presto (now Trino) is solving as well (among other things) ? Even though it is focused only on analytics and not on ML use cases.
[ Parent | on HN ]