News for hackers. Stuff that matters.
Re:Big data is dead (2023) by sdenton4 on Monday May 27, 2024 @10:11AM (#40490882)
Parquet is columnar storage, which is much faster for querying. And typically for protobuf you deserialize each row, which has a performance cost - you need to deserialize the whole message, and can't get just the field you want.

So, of you want to query a giant collection of protobufs, you end up reading and deserializing every record. For parquet, you get much closer to only reading what you need.

Re:Big data is dead (2023) by ddalex on Tuesday May 28, 2024 @03:23AM (#40498268)
Thank you.