News for hackers. Stuff that matters.

A comment on Apache Arrow 3.0. Parent comment

Re:Apache Arrow 3.0 by est on Wednesday February 03, 2021 @10:01PM (#26022095)
Arrow had it selling points as non-serde. But I am wondering how does it achieve no-serde with Python? By allocating PyObject cleverly with a network packet buffer?

If I convert an Arrow int8 array to normal python list of int's, will this involve copying?

[ Parent | on HN ]
Re:Apache Arrow 3.0 by int_19h on Wednesday February 03, 2021 @10:33PM (#26022279)
I think the idea is that you'd have a Python object that behaves exactly like a list of ints would, but with Arrow as its backing store.
[ Parent | on HN ]
Re:Apache Arrow 3.0 by kylebarron on Wednesday February 03, 2021 @11:29PM (#26022590)
This is similar to Numpy. You operate on Python objects that describe the Numpy structure, but the actual memory is stored in C objects, not Python objects.
[ Parent | on HN ]
Re:Apache Arrow 3.0 by dahinds on Wednesday February 03, 2021 @10:54PM (#26022395)
It would have to, but to a numpy vector, maybe not.
[ Parent | on HN ]