News for hackers. Stuff that matters.
Re:Big data is dead (2023) by sph on Monday May 27, 2024 @09:45AM (#40490653)
Third consecutive time in 86 days that you mention .parquet files. I am out of my element here, but it's a bit weird
Re:Big data is dead (2023) by ok_computer on Monday May 27, 2024 @10:00AM (#40490772)
Sometimes when people discover or extensively use something they are eager to share in contexts they think are relevant. There is an issue when those contexts become too broad.

3 times across 3 months is hardly astroturfing for big parquet territory.

Re:Big data is dead (2023) by fifilura on Monday May 27, 2024 @09:51AM (#40490694)
FWIW I am the same. I tend to recommend BigQuery and AWS/Athena in various posts. Many times paired with Parquet.

But it is because it makes a lot of things much simpler, and that a lot of people have not realized that. Tooling is moving fast in this space, it is not 2004 anymore.

His arguments are still valid and 86 days is a pretty long time.

Re:Big data is dead (2023) by mrtimo on Monday May 27, 2024 @04:59PM (#40494648)
I've downloaded many csv files that were mal-formatted (extra commas or tabs etc.), or had dates in non-standard formats. Parquet format probably would not have had these issues!
Re:Big data is dead (2023) by swyx on Monday May 27, 2024 @09:18PM (#40496451)
no need to be so suspicious when its an open standard not even linked to a startup?