News for hackers. Stuff that matters.
Re:Apache Arrow 3.0 by TuringTest on Wednesday February 03, 2021 @04:53PM (#26019656)
If it works as a universal intermediate exchange language, it could help standardize connections among disparate systems.

When you have N systems, it takes N^2 translators to build direct connections to transfer data between them; but it only takes N translators if all them can talk the same exchange language.

Re:Apache Arrow 3.0 by waynesonfire on Wednesday February 03, 2021 @05:09PM (#26019815)
can you define what at translator is? I don't understand the complexity you're constructing. I have N systems and they talk protobuf. What's the problem?
Re:Apache Arrow 3.0 by TuringTest on Wednesday February 03, 2021 @05:17PM (#26019903)
By a translator, I mean a library that allows accessing data from different subsystems (either languages or OS processes).

In this case, the advantages are that 1) Arrow is language agnostic, so it's likely that it can be used as a native library in your program and 2) it doesn't copy data to make it accessible to another process, so it saves a lot of marshalling / unmarshalling steps (assuming both sides use data in tabular format, which is typical of data analysis contexts).