News for hackers. Stuff that matters.

A comment on Apache Arrow 3.0. Parent comment

Re:Apache Arrow 3.0 by kristjansson on Wednesday February 03, 2021 @05:26PM (#26020010)
Not GP post, but it might have been better stated as 'eliminating serde overhead'. Arrow's RPC serialization [1] is basically Protobuf, with a whole lot of hacks to eliminate copies on both ends of the wire. So it's still 'serde', but markedly more efficient for large blocks of tabular-ish data.

[1]: https://arrow.apache.org/docs/format/Flight.html

[ Parent | on HN ]
Re:Apache Arrow 3.0 by wesm on Wednesday February 03, 2021 @05:28PM (#26020034)
> Arrow's serialization is Protobuf

Incorrect. Only Arrow Flight embeds the Arrow wire format in a Protocol Buffer, but the Arrow protocol itself does not use Protobuf.

[ Parent | on HN ]
Re:Apache Arrow 3.0 by kristjansson on Wednesday February 03, 2021 @05:50PM (#26020279)
Apologies, off base there. Edited with a pointer to Flight :)
[ Parent | on HN ]