News for hackers. Stuff that matters.

A comment on Apache Arrow 3.0. Parent comment

Re:Apache Arrow 3.0 by mumblemumble on Wednesday February 03, 2021 @05:36PM (#26020119)
So, there are several components to Arrow. One of them transfers data using IPC, and naturally needs to serialize. The other uses shared memory, which eliminates the need for serde.

Sadly, the latter isn't (yet) well supported anywhere but Python and C++. If you can/do use it, though, data are just kept as as arrays in memory. Which is exactly what the CPU wants to see.

[ Parent | on HN ]
Re:Apache Arrow 3.0 by superdimwit on Wednesday February 03, 2021 @08:21PM (#26021454)
Shared memory format is supported in Julia too!
[ Parent | on HN ]
Re:Apache Arrow 3.0 by mumblemumble on Wednesday February 03, 2021 @08:44PM (#26021594)
Oh, that's fantastic to hear. Right now I'm living in Python because that's the galactic center, but I've also been anxious to find a low-cost escape hatch that doesn't just lead to C++.
[ Parent | on HN ]
Re:Apache Arrow 3.0 by oscardssmith on Wednesday February 03, 2021 @11:37PM (#26022655)
You should definitely check out Julia then. There are a few parts of the language that use C/C++ libraries (blas and mpfr are the main ones), but 95% of the time, your stack will be Julia all the way down.
[ Parent | on HN ]