News for hackers. Stuff that matters.
Re:Apache Arrow 1.0 by mumblemumble on Tuesday July 28, 2020 @08:53AM (#23975156)
Considering the number of segfaults I've witnessed that appear to originate in relatively recent versions of the Parquet library, I don't think your instincts are misplaced. I genuinely worry about data corruption.

I've been meaning to take a closer look at ORC. As a Spark user, I just sort of defaulted into Parquet. ORC is very similar, though, and seemingly gives every indication of being the more mature product.

Re:Apache Arrow 1.0 by cheez on Tuesday July 28, 2020 @11:07AM (#23976565)
On balance I find writing your own tools is useful when your use cases are narrow but becoming a wizard in other tools is useful otherwise. I'm lucky that I get to define my own use cases.