Re:Apache Arrow and MinIO
by anderspitman on Sunday September 20, 2020 @07:28PM
(#24538311)
Is Arrow data commonly written to disk? I was under the impression (as someone who doesn't use it currently) it's usually converted to Parquet for this purpose.
[ Parent
| on HN ]
Arrow is a memory format and optimized for efficient vectorized processing in memory. Although it is possible to persist the Arrow format to disk, it is more common to use Parquet.
Re:Apache Arrow and MinIO
by thundergolfer on Sunday September 20, 2020 @08:24PM
(#24538599)
I'd be interested to know any benefits of converting to Parquet if you know your reader supports Arrow.